Pulumi Blog: Yaml

New: Helm Chart v4 resource with new features and languages

Eron Wright — Thu, 13 Jun 2024 00:00:00 -0700

Today we’re happy to announce a new “v4” version of the Chart resource, available now in v4.13 of the Pulumi Kubernetes provider. The new kubernetes.helm.sh/v4.Chart resource is provided side-by-side with the existing kubernetes.helm.sh/v3.Chart resource. We expect to deprecate v3 in the future.

When you need to install a third-party application into your Kubernetes cluster, you’re likely to find a Helm chart for that in Artifact Hub or other registry. Pulumi provides two ways to apply a Helm chart, as outlined in Choosing the Right Helm Resource For Your Use Case. The Chart resource offers deeper integration with Pulumi and better drift remediation. v4 brings a host of new features, including enhanced SDK support across all Pulumi SDKs, full OCI registry support, improved handling of chart values, better connectivity for cluster interactions, and improved resource ordering. Let’s dig in.

What’s new

Let’s look at what’s new with Chart v4.

New language support - Java SDK & YAML SDK

The Chart v4 resource is a Pulumi Component that works consistently across all Pulumi SDKs. Earlier versions were implemented separately for each SDK, leading to various inconsistencies and limited SDK support.

Here, for example, is a simple deployment of cert-manager, a well-known Kubernetes add-on:

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const certman = new kubernetes.helm.v4.Chart("cert-manager", {
 namespace: "cert-manager",
 chart: "oci://registry-1.docker.io/bitnamicharts/cert-manager",
 version: "1.3.1",
});

import pulumi
import pulumi_kubernetes as kubernetes

certman = kubernetes.helm.v4.Chart("cert-manager",
 namespace="cert-manager",
 chart="oci://registry-1.docker.io/bitnamicharts/cert-manager",
 version="1.3.1")

package main

import (
 helmv4 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v4"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := helmv4.NewChart(ctx, "cert-manager", &helmv4.ChartArgs{
 Namespace: pulumi.String("cert-manager"),
 Chart: pulumi.String("oci://registry-1.docker.io/bitnamicharts/cert-manager"),
 Version: pulumi.String("1.3.1"),
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var certman = new Kubernetes.Helm.V4.Chart("cert-manager", new()
 {
 Namespace = "cert-manager",
 Chart = "oci://registry-1.docker.io/bitnamicharts/cert-manager",
 Version = "1.3.1",
 });

});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.kubernetes.helm.sh_v4.Chart;
import com.pulumi.kubernetes.helm.sh_v4.ChartArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var certman = new Chart("cert-manager", ChartArgs.builder()
 .namespace("cert-manager")
 .chart("oci://registry-1.docker.io/bitnamicharts/cert-manager")
 .version("1.3.1")
 .build());

 }
}

resources:
 cert-manager:
 type: kubernetes:helm.sh/v4:Chart
 properties:
 namespace: cert-manager
 chart: oci://registry-1.docker.io/bitnamicharts/cert-manager
 version: "1.3.1"

OCI registry support

You can use container registries with OCI support such as Docker Hub to store and share Helm chart packages. The Chart v4 resource now has full support for OCI, bringing it to parity with the Release resource.

To use an authenticated OCI registry, you must first login using helm registry login or docker login.

Chart v4 also supports the use of Helm chart repositories, and adopts the same repositoryOpts API as was introduced in the Release resource.

Lock file support

Helm has support for lock files (Chart.lock) to control a chart’s dependencies. When deploying a chart from a local directory, Pulumi automatically rebuilds the chart’s dependencies if a lock file is present. See Helm Dependency Build for details.

Better handling of chart values

Chart v4 offers new ways to work with Chart values. It is now possible to use multiple values files and to use Pulumi Assets. Of course you can also use output values from other resources as chart values.

It is also possible to set a chart value to the contents of a text file, similarly to using Helm’s --set-file argument. To do that, simply use a Pulumi Asset as a value within the values map. This is useful for injecting large values into a chart, such as additional templates as supported by the extraDeploy parameter of some Bitnami charts.

Here’s an example of using a combination of values from different sources.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const certManager = new kubernetes.helm.v4.Chart("cert-manager", {
 namespace: "cert-manager",
 chart: "oci://registry-1.docker.io/bitnamicharts/cert-manager",
 version: "1.3.1",
 valueYamlFiles: [new pulumi.asset.FileAsset("values.yaml")],
 values: {
 commonLabels: {
 "pulumi.com/stackName": pulumi.getStack(),
 },
 extraDeploy: [new pulumi.asset.FileAsset("manifest.yaml")],
 },
});

import pulumi
import pulumi_kubernetes as kubernetes

cert_manager = kubernetes.helm.v4.Chart("cert-manager",
 namespace="cert-manager",
 chart="oci://registry-1.docker.io/bitnamicharts/cert-manager",
 version="1.3.1",
 value_yaml_files=[pulumi.FileAsset("values.yaml")],
 values={
 "commonLabels": {
 "pulumi.com/stackName": pulumi.get_stack(),
 },
 "extraDeploy": [pulumi.FileAsset("manifest.yaml")],
 })

package main

import (
 helmv4 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v4"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := helmv4.NewChart(ctx, "cert-manager", &helmv4.ChartArgs{
 Namespace: pulumi.String("cert-manager"),
 Chart: pulumi.String("oci://registry-1.docker.io/bitnamicharts/cert-manager"),
 Version: pulumi.String("1.3.1"),
 ValueYamlFiles: pulumi.AssetOrArchiveArray{
 pulumi.NewFileAsset("values.yaml"),
 },
 Values: pulumi.Map{
 "commonLabels": pulumi.Any(map[string]interface{}{
 "pulumi.com/stackName": ctx.Stack(),
 }),
 "extraDeploy": pulumi.Array{
 pulumi.NewFileAsset("manifest.yaml"),
 },
 },
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var certManager = new Kubernetes.Helm.V4.Chart("cert-manager", new()
 {
 Namespace = "cert-manager",
 Chart = "oci://registry-1.docker.io/bitnamicharts/cert-manager",
 Version = "1.3.1",
 ValueYamlFiles = new[]
 {
 new FileAsset("values.yaml"),
 },
 Values =
 {
 { "commonLabels", new Dictionary<string, object?>
 {
 ["pulumi.com/stackName"] = Deployment.Instance.StackName,
 } },
 { "extraDeploy", new[]
 {
 new FileAsset("manifest.yaml"),
 } },
 },
 });

});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.deployment.Deployment;
import com.pulumi.kubernetes.helm.v4.Chart;
import com.pulumi.kubernetes.helm.v4.ChartArgs;
import com.pulumi.asset.FileAsset;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var certManager = new Chart("cert-manager", ChartArgs.builder()
 .namespace("cert-manager")
 .chart("oci://registry-1.docker.io/bitnamicharts/cert-manager")
 .version("1.3.1")
 .valueYamlFiles(new FileAsset("values.yaml"))
 .values(Map.ofEntries(
 Map.entry("commonLabels", Map.of("pulumi.com/stackName", Deployment.getInstance().getStackName())),
 Map.entry("extraDeploy", List.of(new FileAsset("manifest.yaml")))
 ))
 .build());

 }
}

resources:
 cert-manager:
 type: kubernetes:helm.sh/v4:Chart
 properties:
 namespace: cert-manager
 chart: oci://registry-1.docker.io/bitnamicharts/cert-manager
 version: "1.3.1"
 valueYamlFiles:
 - fn::fileAsset: values.yaml
 values:
 commonLabels:
 pulumi.com/stackName: ${pulumi.stack}
 extraDeploy:
 - fn::fileAsset: manifest.yaml

Better connectivity

You may now use charts that use chart functions requiring a connection to the cluster, e.g. to:

Check the Kubernetes server version with .Capabilities.KubeVersion
Check if an API version or kind is available with .Capabilities.APIVersions.Has
Use the lookup function in your templates to get existing objects in your live cluster.

Note that the lookup function is executed in both the preview and the non-preview mode, and keep in mind that the expected object may not exist during a preview.

Improved resource ordering

It’s now easy to wait for a chart’s resources to be installed before installing other resources, simply by using the dependsOn option. In earlier versions, we relied on a ready output property.

The Chart resource automatically detects dependencies between resources in the manifest(s). For example, it knows to install namespaces and Custom Resource Definitions (CRDs) first.

Use the config.kubernetes.io/depends-on annotation to declare an explicit resource dependency. The annotation accepts a list of resource references, delimited by commas.

It consists of the group, kind, name, and optionally the namespace, delimited by forward slashes.

Resource Scope	Format
namespace-scoped	`<group>/namespaces/<namespace>/<kind>/<name>`
cluster-scoped	`<group>/<kind>/<name>`

For resources in the “core” group, the empty string is used instead (for example: /namespaces/test/Pod/pod-a).

New-style Pulumi transformations

Pulumi has a new way to transform component resources and their children, the transforms options. The older transformations option doesn’t work with multi-language components like Chart v4. See Resource Option: transforms for more details.

Note: you cannot change an object’s namespace or name using a Pulumi transformation, and you cannot add or discard an object.

Here’s an example of using the transforms option to add the pulumi.com/patchForce annotation to a chart’s resources.

const applyPatchForceAnnotation = async (args: pulumi.ResourceTransformArgs) => {
 switch(args.type) {
 case "kubernetes:helm.sh/v4:Chart":
 break;
 default:
 args.props.metadata.annotations = {
 "pulumi.com/patchForce": "true",
 ...args.props.metadata.annotations
 }
 }
 return {
 props: args.props,
 opts: args.opts,
 };
};

const ingressController = new kubernetes.helm.v4.Chart("ingresscontroller", {
 chart: "nginx-ingress",
 namespace: ingressNs.metadata.name,
 repositoryOpts: {
 repo: "https://helm.nginx.com/stable",
 },
 version: "0.14.1",
}, {transforms: [applyPatchForceAnnotation]});

def apply_patchforce_annotation(args: ResourceTransformArgs):
 if not args.type_ == "kubernetes:helm.sh/v4:Chart":
 if not 'metadata' in args.props:
 args.props['metadata'] = {}
 if not 'annotations' in args.props['metadata']:
 args.props['metadata']['annotations'] = {}
 args.props['metadata']['annotations']['pulumi.com/patchForce'] = 'true'

 return ResourceTransformResult(
 props=args.props,
 opts=args.opts)

ingresscontroller = kubernetes.helm.v4.Chart(
 "ingresscontroller",
 chart="nginx-ingress",
 namespace=ingress_ns.metadata.name,
 repository_opts=kubernetes.helm.v3.RepositoryOptsArgs(
 repo="https://helm.nginx.com/stable",
 ),
 version="0.14.1",
 opts=pulumi.ResourceOptions(transforms=[apply_patchforce_annotation])
)

applyPatchForceAnnotation := func(ctx context.Context, rta *pulumi.ResourceTransformArgs) *pulumi.ResourceTransformResult {
 transform := func(applier interface{}) {
 o := rta.Props.ToMapOutputWithContext(ctx).ApplyT(applier)
 r, err := internals.UnsafeAwaitOutput(ctx, o)
 if err != nil {
 panic(err)
 }
 rta.Props = r.Value.(pulumi.Map)
 }

 switch rta.Type {
 case "kubernetes:helm.sh/v4:Chart":
 // Do nothing for Helm charts
 default:
 transform(func(obj map[string]any) pulumi.Map {
 // note: obj is an ordinary Unstructured object at this point.
 unstructured.SetNestedField(obj, "true", "metadata", "annotations", "pulumi.com/patchForce")
 return pulumi.ToMap(obj)
 })
 }
 return &pulumi.ResourceTransformResult{
 Props: rta.Props,
 Opts: rta.Opts,
 }
}

// Use Helm to install the Nginx ingress controller
_, err = helmv4.NewChart(ctx, "ingresscontroller", &helmv4.ChartArgs{
 Chart: pulumi.String("nginx-ingress"),
 Namespace: ingressNs.Metadata.Name(),
 RepositoryOpts: &helmv4.RepositoryOptsArgs{
 Repo: pulumi.String("https://helm.nginx.com/stable"),
 },
 Version: pulumi.String("0.14.1"),
}, pulumi.Transforms([]pulumi.ResourceTransform{applyPatchForceAnnotation}))
if err != nil {
 return err
}

Post-rendering support

New to v4 is support for a post-rendering command, with optional arguments, to be applied to the rendered manifest. See Advanced Helm Techniques: Post Rendering for details.

“Keep” policy

The Chart v4 resource now understands Helm resource policies, specifically “keep” which instructs Pulumi not to delete a given object when the resource is destroyed. Simply apply the helm.sh/resource-policy: keep annotation to the object. See Tell Helm Not To Uninstall a Resource for details.

Release name

Charts typically generate object names based on the Helm release name. By default, Chart v4 uses its own resource name as the release name. To customize the release name, set the name property on the Chart.

Some charts also accept a fullnameOverride value to control object naming, use that if possible.

Limitations

Not supported: Kubernetes transformations

Chart v4 does not support the transformations argument as seen in Chart v3, that facilitates a Kubernetes-centric transformation and/or discarding of objects from the rendered manifest.

One alternative is to use use Pulumi transformations to transform the object and resource options. Another is to use post-rendering, which we’ll cover next.

Not supported: Helm hooks

Some charts use hooks to perform pre- and post-installation tasks, and we recommend using the Release resource to install such charts. Another option is to emulate the hook by performing the task directly, using other Pulumi resources such as kubernetes.batch/v1.Job and kubernetes.core/v1.Secret.

Note that Chart v3 does not support hooks either but, for historical reasons, does emit the hooks as ordinary child resources.

Evolving: Resource outputs

The child resources created by the chart are presented to your program via the resources output, as an array of Pulumi Resource objects. In some SDKs, you can cast the resource into the appropriate resource type, e.g. corev1.ConfigMap. Not all SDKs support this (yet); see pulumi/pulumi#15788.

Deploy Helm charts with Pulumi

Apply any Helm chart with the Chart v4 resource and manage the cloud resources it depends on, like IAM roles and storage buckets, in the same program.

Get started

Example: Argo CD

Here’s a real-world example of installing Argo CD into a Kubernetes cluster, and of using Argo CD’s Application custom resource to deploy the ‘guestbook’ example.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";
import * as random from "@pulumi/random";

const ns = new kubernetes.core.v1.Namespace("ns", {metadata: {
 name: "argocd",
}});
const redisPasswordResource = new random.RandomPassword("redis-password", {length: 16});
const redisSecret = new kubernetes.core.v1.Secret("redis-secret", {
 metadata: {
 name: "argocd-redis",
 namespace: ns.metadata.apply(metadata => metadata.name),
 },
 type: "Opaque",
 stringData: {
 auth: redisPasswordResource.result,
 },
});
const argocd = new kubernetes.helm.v4.Chart("argocd", {
 chart: "argo-cd",
 version: "6.11.1",
 namespace: ns.metadata.apply(metadata => metadata.name),
 repositoryOpts: {
 repo: "https://argoproj.github.io/argo-helm",
 },
 values: {
 fullnameOverride: "",
 },
});
const guestbook = new kubernetes.yaml.v2.ConfigGroup("guestbook", {objs: [{
 apiVersion: "argoproj.io/v1alpha1",
 kind: "Application",
 metadata: {
 name: "guestbook",
 namespace: ns.metadata.apply(metadata => metadata.name),
 },
 spec: {
 project: "default",
 source: {
 repoURL: "https://github.com/argoproj/argocd-example-apps.git",
 targetRevision: "HEAD",
 path: "guestbook",
 },
 destination: {
 server: "https://kubernetes.default.svc",
 namespace: "default",
 },
 },
}]}, {
 dependsOn: [argocd],
});
export const redisPassword = redisPasswordResource.result;

import pulumi
import pulumi_kubernetes as kubernetes
import pulumi_random as random

ns = kubernetes.core.v1.Namespace("ns", metadata=kubernetes.meta.v1.ObjectMetaArgs(
 name="argocd",
))
redis_password_resource = random.RandomPassword("redis-password", length=16)
redis_secret = kubernetes.core.v1.Secret("redis-secret",
 metadata=kubernetes.meta.v1.ObjectMetaArgs(
 name="argocd-redis",
 namespace=ns.metadata.name,
 ),
 type="Opaque",
 string_data={
 "auth": redis_password_resource.result,
 })
argocd = kubernetes.helm.v4.Chart("argocd",
 chart="argo-cd",
 version="6.11.1",
 namespace=ns.metadata.name,
 repository_opts=kubernetes.helm.v4.RepositoryOptsArgs(
 repo="https://argoproj.github.io/argo-helm",
 ),
 values={
 "fullnameOverride": "",
 })
guestbook = kubernetes.yaml.v2.ConfigGroup("guestbook", objs=[{
 "apiVersion": "argoproj.io/v1alpha1",
 "kind": "Application",
 "metadata": {
 "name": "guestbook",
 "namespace": ns.metadata.name,
 },
 "spec": {
 "project": "default",
 "source": {
 "repoURL": "https://github.com/argoproj/argocd-example-apps.git",
 "targetRevision": "HEAD",
 "path": "guestbook",
 },
 "destination": {
 "server": "https://kubernetes.default.svc",
 "namespace": "default",
 },
 },
}],
opts=pulumi.ResourceOptions(depends_on=[argocd]))
pulumi.export("redisPassword", redis_password_resource.result)

package main

import (
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 helmv4 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v4"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 yamlv2 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/yaml/v2"
 "github.com/pulumi/pulumi-random/sdk/v4/go/random"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 ns, err := corev1.NewNamespace(ctx, "ns", &corev1.NamespaceArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("argocd"),
 },
 })
 if err != nil {
 return err
 }
 redisPasswordResource, err := random.NewRandomPassword(ctx, "redis-password", &random.RandomPasswordArgs{
 Length: pulumi.Int(16),
 })
 if err != nil {
 return err
 }
 _, err = corev1.NewSecret(ctx, "redis-secret", &corev1.SecretArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("argocd-redis"),
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Type: pulumi.String("Opaque"),
 StringData: pulumi.StringMap{
 "auth": redisPasswordResource.Result,
 },
 })
 if err != nil {
 return err
 }
 argocd, err := helmv4.NewChart(ctx, "argocd", &helmv4.ChartArgs{
 Chart: pulumi.String("argo-cd"),
 Version: pulumi.String("6.11.1"),
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 RepositoryOpts: &helmv4.RepositoryOptsArgs{
 Repo: pulumi.String("https://argoproj.github.io/argo-helm"),
 },
 Values: pulumi.Map{
 "fullnameOverride": pulumi.Any(""),
 },
 })
 if err != nil {
 return err
 }
 _, err = yamlv2.NewConfigGroup(ctx, "guestbook", &yamlv2.ConfigGroupArgs{
 Objs: pulumi.Array{
 pulumi.Any(map[string]interface{}{
 "apiVersion": "argoproj.io/v1alpha1",
 "kind": "Application",
 "metadata": map[string]interface{}{
 "name": "guestbook",
 "namespace": ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 "spec": map[string]interface{}{
 "project": "default",
 "source": map[string]interface{}{
 "repoURL": "https://github.com/argoproj/argocd-example-apps.git",
 "targetRevision": "HEAD",
 "path": "guestbook",
 },
 "destination": map[string]interface{}{
 "server": "https://kubernetes.default.svc",
 "namespace": "default",
 },
 },
 }),
 },
 }, pulumi.DependsOn([]pulumi.Resource{
 argocd,
 }))
 if err != nil {
 return err
 }
 ctx.Export("redisPassword", redisPasswordResource.Result)
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;
using Random = Pulumi.Random;

return await Deployment.RunAsync(() =>
{
 var ns = new Kubernetes.Core.V1.Namespace("ns", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "argocd",
 },
 });

 var redisPasswordResource = new Random.RandomPassword("redis-password", new()
 {
 Length = 16,
 });

 var redisSecret = new Kubernetes.Core.V1.Secret("redis-secret", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "argocd-redis",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Type = "Opaque",
 StringData =
 {
 { "auth", redisPasswordResource.Result },
 },
 });

 var argocd = new Kubernetes.Helm.V4.Chart("argocd", new()
 {
 Chart = "argo-cd",
 Version = "6.11.1",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 RepositoryOpts = new Kubernetes.Types.Inputs.Helm.V4.RepositoryOptsArgs
 {
 Repo = "https://argoproj.github.io/argo-helm",
 },
 Values =
 {
 { "fullnameOverride", "" },
 },
 });

 var guestbook = new Kubernetes.Yaml.V2.ConfigGroup("guestbook", new()
 {
 Objs = new[]
 {
 new Dictionary<string, object?>
 {
 ["apiVersion"] = "argoproj.io/v1alpha1",
 ["kind"] = "Application",
 ["metadata"] = new Dictionary<string, object?>
 {
 ["name"] = "guestbook",
 ["namespace"] = ns.Metadata.Apply(metadata => metadata.Name),
 },
 ["spec"] = new Dictionary<string, object?>
 {
 ["project"] = "default",
 ["source"] = new Dictionary<string, object?>
 {
 ["repoURL"] = "https://github.com/argoproj/argocd-example-apps.git",
 ["targetRevision"] = "HEAD",
 ["path"] = "guestbook",
 },
 ["destination"] = new Dictionary<string, object?>
 {
 ["server"] = "https://kubernetes.default.svc",
 ["namespace"] = "default",
 },
 },
 },
 },
 }, new ComponentResourceOptions
 {
 DependsOn =
 {
 argocd,
 },
 });

 return new Dictionary<string, object?>
 {
 ["redisPassword"] = redisPasswordResource.Result,
 };
});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.kubernetes.core.v1.Namespace;
import com.pulumi.kubernetes.core.v1.NamespaceArgs;
import com.pulumi.kubernetes.meta.v1.inputs.ObjectMetaArgs;
import com.pulumi.random.RandomPassword;
import com.pulumi.random.RandomPasswordArgs;
import com.pulumi.kubernetes.core.v1.Secret;
import com.pulumi.kubernetes.core.v1.SecretArgs;
import com.pulumi.kubernetes.helm.v4.Chart;
import com.pulumi.kubernetes.helm.v4.ChartArgs;
import com.pulumi.kubernetes.helm.v4.inputs.RepositoryOptsArgs;
import com.pulumi.kubernetes.yaml.v2.ConfigGroup;
import com.pulumi.kubernetes.yaml.v2.ConfigGroupArgs;
import com.pulumi.resources.ComponentResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var ns = new Namespace("ns", NamespaceArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("argocd")
 .build())
 .build());

 var redisPasswordResource = new RandomPassword("redisPasswordResource", RandomPasswordArgs.builder()
 .length(16)
 .build());

 var redisSecret = new Secret("redisSecret", SecretArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("argocd-redis")
 .namespace(ns.metadata().applyValue(metadata -> metadata.name().get()))
 .build())
 .type("Opaque")
 .stringData(redisPasswordResource.result().applyValue(password -> Map.of("auth", password)))
 .build());

 var argocd = new Chart("argocd", ChartArgs.builder()
 .chart("argo-cd")
 .version("6.11.1")
 .namespace(ns.metadata().applyValue(metadata -> metadata.name().get()))
 .repositoryOpts(RepositoryOptsArgs.builder()
 .repo("https://argoproj.github.io/argo-helm")
 .build())
 .values(Map.of("fullnameOverride", ""))
 .build());

 var guestbook = new ConfigGroup("guestbook", ConfigGroupArgs.builder()
 .objs(Map.ofEntries(
 Map.entry("apiVersion", "argoproj.io/v1alpha1"),
 Map.entry("kind", "Application"),
 Map.entry("metadata", Map.ofEntries(
 Map.entry("name", "guestbook"),
 Map.entry("namespace", ns.metadata().applyValue(metadata -> metadata.name().get()))
 )),
 Map.entry("spec", Map.ofEntries(
 Map.entry("project", "default"),
 Map.entry("source", Map.ofEntries(
 Map.entry("repoURL", "https://github.com/argoproj/argocd-example-apps.git"),
 Map.entry("targetRevision", "HEAD"),
 Map.entry("path", "guestbook")
 )),
 Map.entry("destination", Map.ofEntries(
 Map.entry("server", "https://kubernetes.default.svc"),
 Map.entry("namespace", "default")
 ))
 ))
 ))
 .build(), ComponentResourceOptions.builder()
 .dependsOn(argocd)
 .build());

 ctx.export("redisPassword", redisPasswordResource.result());
 }
}

resources:
 ns:
 type: kubernetes:core/v1:Namespace
 properties:
 metadata:
 name: argocd

 redis-password:
 type: random:RandomPassword
 properties:
 length: 16

 redis-secret:
 type: kubernetes:core/v1:Secret
 properties:
 metadata:
 name: argocd-redis
 namespace: ${ns.metadata.name}
 type: Opaque
 stringData:
 auth: ${redis-password.result}
 options:
 retainOnDelete: true

 argocd:
 type: kubernetes:helm.sh/v4:Chart
 properties:
 chart: argo-cd
 version: "6.11.1"
 namespace: ${ns.metadata.name}
 repositoryOpts:
 repo: https://argoproj.github.io/argo-helm
 values:
 fullnameOverride: ""

 guestbook:
 type: kubernetes:yaml/v2:ConfigGroup
 properties:
 objs:
 - apiVersion: argoproj.io/v1alpha1
 kind: Application
 metadata:
 name: guestbook
 namespace: ${ns.metadata.name}
 spec:
 project: default
 source:
 repoURL: https://github.com/argoproj/argocd-example-apps.git
 targetRevision: HEAD
 path: guestbook
 destination:
 server: https://kubernetes.default.svc
 namespace: default
 options:
 dependsOn:
 - ${argocd}

outputs:
 redisPassword: ${redis-password.result}

The program creates the argocd namespace, installs the ArgoCD server, and then creates an ArgoCD Application resource. Observe how the program installs and uses a Custom Resource Definition (CRD) successfully, and uses dependsOn to ensure that the CRD is installed before using it.

The argo-cd chart normally makes use of a Helm hook to initialize a password for the redis server. Since the Chart v4 resource doesn’t support Helm hooks, this program creates the password directly.

Conclusion

Pulumi loves empowering developers to use the best tools for the job, and we recommend using Helm charts to install third-party Kubernetes applications. Pulumi complements Helm by handling the cloud resources that are often required, such as an IAM Role or cloud storage bucket. Such combinations make for great reusable componentry.

Check out the following links to learn more about Pulumi Kubernetes today!

New: ConfigGroup, ConfigFile resources for Java, YAML SDKs

Eron Wright — Fri, 12 Apr 2024 00:00:00 -0700

The Pulumi Kubernetes provider makes it easy to deploy Kubernetes resources to your cluster, giving you options based on how your application or workload is packaged. The options include strongly-typed resources for standard Kubernetes types, Helm charts, Kustomizations, and Kubernetes manifests.

In v4.10, we leveled up the support for working with Kubernetes manifests with the introduction of the yaml/v2 package. The package provides new implementations of the ConfigGroup and ConfigFile resources, expanding support to the Pulumi Java SDK and to Pulumi YAML. The new implementations are also smarter about applying the objects in the correct order.

Please note that these resources are in a preview stage of maturity, as we continue to round out the feature set. These new resources are provided side-by-side with the existing implementations.

Let’s explore how it’s easier than ever to deploy diverse Kubernetes workloads using Pulumi.

kubernetes.yaml/v2.ConfigGroup

The ConfigGroup resource creates a set of Kubernetes objects from Kubernetes manifests and/or object literals. The Kubernetes manifest may be supplied using any combination of the following methods:

a filename or a list of filenames.
a file pattern or a list of file patterns.
a literal string containing YAML or JSON text.
a literal object or a list of objects.

The ConfigGroup resource accepts the following input properties:

Property	Description
`files`	Set of files, patterns and/or URLs to Kubernetes manifest files. Filenames containing any of the characters ``, `?`, and `[` are treated as patterns*.
`objs`	Set of Kubernetes objects.
`yaml`	A Kubernetes manifest as a YAML or JSON string.
`resourcePrefix`	A prefix for auto-generated names of the child resources. Defaults to the `ConfigGroup` name.
`skipAwait`	Skips over the readiness checks on the child resources.

Note that all objects defined within the manifest must have a metadata.name field; Pulumi autonaming is not supported.

See the API Reference documentation for more information.

Manifest Files

Here’s an example of applying a Kubernetes manifest file to your cluster.

name: demo-cg-1
runtime: yaml
resources:
 cg:
 type: kubernetes:yaml/v2:ConfigGroup
 properties:
 files:
 - ./manifest.yaml

YAML Text

Here’s how to apply an inline manifest.

name: demo-cg-2
runtime: yaml
resources:
 cg:
 type: kubernetes:yaml/v2:ConfigGroup
 properties:
 yaml: |
 apiVersion: v1
 kind: ConfigMap
 metadata:
 name: my-server-config
 data:
 altGreeting: "Good Morning!"
 ---
 apiVersion: v1
 kind: Secret
 metadata:
 name: my-dotfile-secret
 data:
 .secret-file: dmFsdWUtMg0KDQo=

Object Literals

With ConfigGroup it’s easy to define a Kubernetes object using Pulumi property values. Let’s create a Kubernetes namespace and deploy a custom resource into that namespace.

name: demo-cg-3
runtime: yaml
resources:
 ns:
 type: kubernetes:core/v1:Namespace
 cg:
 type: kubernetes:yaml/v2:ConfigGroup
 properties:
 objs:
 - apiVersion: stable.example.com/v1
 kind: CronTab
 metadata:
 namespace: ${ns.metadata.name}
 name: my-crontab
 spec:
 cronSpec: "* * * * */5"

This feature is especially useful for Pulumi YAML, since the apiextensions.k8s.io:CustomResource resource isn’t supported yet (see pulumi-kubernetes#2787).

Resource Ordering

The ConfigGroup resource automatically detects dependencies between resources in the manifest(s). For example, it knows to install namespaces and Custom Resource Definitions (CRDs) first.

Use the config.kubernetes.io/depends-on annotation to declare an explicit resource dependency. The annotation accepts a list of resource references, delimited by commas.

It consists of the group, kind, name, and optionally the namespace, delimited by forward slashes.

Resource Scope	Format
namespace-scoped	`<group>/namespaces/<namespace>/<kind>/<name>`
cluster-scoped	`<group>/<kind>/<name>`

For resources in the “core” group, the empty string is used instead (for example: /namespaces/test/Pod/pod-a).

Resource Prefixes

The purpose of the resource prefix is to ensure the uniqueness of child resource names, as described in Pulumi: Resource Names. The best practice is to use the component name as a prefix, and now that’s the default behavior.

Note that the resource prefix is not applied to the Kubernetes object names.

Transformation Support

The Kubernetes provider generally supports resource transformations, e.g. to apply a resource alias or to apply an object name prefix or object label. Transformations aren’t yet possible with the new ConfigGroup resource (see pulumi/pulumi#12996), but stay tuned as we work towards a GA release later this year.

kubernetes.yaml/v2.ConfigFile

The ConfigFile resource creates a set of Kubernetes objects from a Kubernetes manifest. While ConfigGroup offers a superset of functionality, the ConfigFile gives a concise way to apply a specific manifest.

The ConfigFile resource accepts the following input properties:

Property	Description
`file`	A file or URL to a Kubernetes manifest file.
`resourcePrefix`	A prefix for auto-generated names of the child resources. Defaults to the `ConfigFile` name.
`skipAwait`	Skips over the readiness checks on the child resources.

See the API Reference documentation for more information.

Manifest File

Here’s an example of applying a Kubernetes manifest file to your cluster:

name: demo-cg-1
runtime: yaml
resources:
 cf:
 type: kubernetes:yaml/v2:ConfigFile
 properties:
 file: ./manifest.yaml

Example: Certificate Manager

Here’s a complete example of how to install and use cert-manager using ConfigGroup resources.

name: cert-manager
runtime: yaml
description: Installs cert-manager. See https://cert-manager.io/docs/installation/kubectl/ for details.
variables: {}
resources:
 install:
 type: kubernetes:yaml/v2:ConfigGroup
 properties:
 files:
 - https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml
 test:
 type: kubernetes:yaml/v2:ConfigGroup
 options:
 dependsOn:
 - ${install}
 properties:
 yaml: |
 apiVersion: v1
 kind: Namespace
 metadata:
 name: cert-manager-test
 ---
 apiVersion: cert-manager.io/v1
 kind: Issuer
 metadata:
 name: test-selfsigned
 namespace: cert-manager-test
 spec:
 selfSigned: {}
 ---
 apiVersion: cert-manager.io/v1
 kind: Certificate
 metadata:
 name: selfsigned-cert
 namespace: cert-manager-test
 annotations:
 config.kubernetes.io/depends-on: cert-manager.io/namespaces/cert-manager-test/Issuer/test-selfsigned
 spec:
 dnsNames:
 - example.com
 secretName: selfsigned-cert-tls
 issuerRef:
 name: test-selfsigned

Conclusion

With the new resources in yaml/v2, we’re excited to enable new scenarios in Pulumi YAML and the Pulumi Java SDK. Stay tuned for new editions of the Chart and Directory resources, coming soon.

This is part of a broad initiative to use Multi-Language Component (MLC) technology to offer a consistent experience across the Pulumi SDKs. See pulumi-kubernetes#1971 for details.

Check out the following links to learn more about Pulumi Kubernetes today!

How to Create and Share a Pulumi Template

Christian Nunciato — Mon, 12 Dec 2022 00:00:00 +0000

Last month, we released our first set of architecture templates — configurable Pulumi projects designed to make it easy to bootstrap new stacks for common cloud architectures like static websites, containers, virtual machines, and Kubernetes clusters. Architecture templates are a great way to get a new project up and running quickly, and they’ve already grown quite popular with our users, several of whom have asked if whether it’s possible to create templates of their own.

It’s not only possible, it’s surprisingly easy! So in this post, I’ll show you how. It all starts with a Pulumi project — a template is really just a Pulumi project with a few extra lines of config — so we’ll start with one of those, add a few resources, and turn the project into a template. When it’s ready to go, we’ll test it out locally, and then finally, we’ll publish it so that anyone with Pulumi can use our template to kickstart their own projects.

We’ll do this in five steps:

Create a new project
Design and build the program
Turn the project into a template
Test-drive the template locally
Publish the template to share it with the world

Let’s get to it.

Step 1: Create a new project

If you haven’t already, make sure you’ve installed Pulumi and configured your AWS credentials. For this example, we’ll be using AWS, but the process is the same for any cloud provider, so if you aren’t set up on AWS, you should still be able to follow along anyway.

Create a new project in the usual way using one of our starter templates. The YAML template is a good choice for this walkthrough, as YAML projects are simple and lightweight (no runtime dependencies!), and because they combine both project and program into one file, they’re especially conducive to sharing:

$ mkdir my-template-project && cd my-template-project
$ pulumi new aws-yaml

Step through the prompts, choosing whichever AWS region works best for you. When the new-project wizard finishes, open the generated Pulumi.yaml project file in your editor of choice to start building.

Step 2: Design and build the program

I’m a static-website person, myself — I think pretty much everyone should have one. So in order to realize my grand vision of home page for every human, you and I are going to build a Pulumi project template that lets anyone with the Pulumi CLI deploy a super-simple website of their own on AWS.

As Pulumi template authors, our general goal is twofold:

Make it easy for our users to create deployable projects that address an infrastructural need.
Make it easy to customize and extend those projects after they’ve been created.

A good template, in other words, is one that not only takes you from pulumi new to pulumi up in as few steps as possible, but that also leaves you with an open, extensible program you can use as a foundation to build upon. For us, the goal is to create a template that can be used to provision the minimal set of cloud resources one needs to run a static website on AWS. And done well, our template will also lend itself easily to further development — custom domains, serverless functions, edge caching, and so on.

To that end, let’s start by replacing the contents of Pulumi.yaml with the following program, which defines the core resources we need: an S3 bucket to hold the files of the website and an S3 bucket object (index.html) to serve as its home page. We’ll also export the computed URL of the website as a Pulumi stack output to give us something to navigate to after deployment:

name: my-template-project
description: A simple static website on Amazon S3
runtime: yaml
resources:

 # Create an S3 bucket and configure it as a website.
 bucket:
 type: aws:s3:Bucket
 properties:
 acl: public-read
 website:
 indexDocument: index.html

 # Create a home page for the website.
 index.html:
 type: aws:s3:BucketObject
 properties:
 bucket: ${bucket.id}
 acl: public-read
 contentType: text/html
 content: |
 <html>
 <meta name="content" charset="utf-8">
 <title>My Awesome Website</title>
 <body>
 <h1>My Awesome Website</h1>
 <p>Hello, internet person! 👋</p>
 </body>
 </html>

# Export the URL of the website.
outputs:
 url: http://${bucket.websiteEndpoint}

Try running this program now, just to make sure everything works. In a few seconds, you should see that the new resources were created and be able to browse to the new website at the exported url:

$ pulumi up
...
Updating (dev)
Type Name Status
+ pulumi:pulumi:Stack my-s3-website-template-dev created (3s)
+ ├─ aws:s3:Bucket bucket created (2s)
+ └─ aws:s3:BucketObject index.html created (0.37s)
Outputs:
url: "http://bucket-f1bb181.s3-website-us-west-2.amazonaws.com"
Resources:
+ 3 created
Duration: 5s

Good — we have a working program. And looking back at our design goals, you’ll see that this program already meets them pretty well:

It works out of the box. Deployed now, it produces a running website on AWS.
It’s open and extensible. To build onto it — add more pages, attach a custom domain, etc. — you’d simply add more resources alongside those already defined.

Now let’s turn this program into a template.

Before moving on, though, let’s quickly destroy and remove this stack, as we’ll be making a few changes to the project’s settings in the next step, and it’d be better to begin from a clean slate:

$ pulumi destroy --yes --remove

Step 3: Turn the project into a template

The next thing to do is decide which parts of the program should be configurable. Earlier, when you ran pulumi new aws-yaml, you probably noticed you were prompted for an optional AWS region:

$ pulumi new aws-yaml
...
This command will walk you through creating a new Pulumi project.
...
aws:region: The AWS region to deploy into: (us-west-2)

You got this prompt because the authors of the aws-yaml template knew that not every user would want to deploy into the same hard-coded AWS region, so they defined an aws:region setting to make it both configurable and optional, falling back to us-west-2 by default. Users of this template are free to change this value if they like or leave it alone and accept the default.

This is all made possible by the existence of the template block in Pulumi.yaml:

# The `template` block from the aws-yaml template.
template:
 description: A minimal AWS Pulumi YAML program
 config:
 aws:region:
 description: The AWS region to deploy into
 default: us-west-2

Indeed, this block is all that defines aws-yaml as a template; without it, the aws-yaml project would be a regular ol’ Pulumi YAML program like any other.

The template block defines two properties:

An optional description property to give new projects created from the template
A config block that lists the names, descriptions, and default values of any settings that should be configurable for new projects

The template block is essentially where all the new-project magic happens. Any settings you define in this block will prompt users for their values and apply those settings to the project’s initial stack (for example, the dev stack we created in Step 1). By default, these values are captured and applied as plain-text strings, but you can also capture them as encrypted Pulumi secrets by adding secret: true — an appropriate choice for prompting for sensitive data like passwords, API keys, and the like.

Our super-simple website template doesn’t need much in the way of configurability — but let’s define some anyway just to see how it’s done. While we’re at it, we’ll make a few adjustments to make project names and descriptions configurable as well.

Make the changes below to Pulumi.yaml to use the project name and description supplied by the CLI, add a template section to let users configure a home-page title and greeting, and interpolate those values into the body of the home page itself. Here’s the full content of the program for reference:

# Configure project names and descriptions with values obtained from `pulumi new`.
name: ${PROJECT}
description: ${DESCRIPTION}
runtime: yaml

# Define the template's configuration settings.
template:
 description: A simple static website on Amazon S3
 config:
 aws:region:
 description: The AWS region to deploy into
 default: us-west-2
 title:
 description: A title to use for the home page
 default: My Awesome Website
 greeting:
 description: A greeting to show on the home page
 default: Hello, internet person! 👋

resources:

 # Create an S3 bucket and configure it as a website.
 bucket:
 type: aws:s3:Bucket
 properties:
 acl: public-read
 website:
 indexDocument: index.html

 # Create a home page for the website.
 index.html:
 type: aws:s3:BucketObject
 properties:
 bucket: ${bucket.id}
 acl: public-read
 contentType: text/html
 content: |
 <html>
 <meta name="content" charset="utf-8">
 <title>${title}</title>
 <body>
 <h1>${title}</h1>
 <p>${greeting}</p>
 </body>
 </html>

# Export the URL of the website.
outputs:
 url: http://${bucket.websiteEndpoint}

That’s all there is to it! The template’s complete and ready to be published. However, before we do that, let’s first give it a try locally just to make sure it delivers the new-project experience we’re looking for.

Step 4: Test-drive the template locally

Back in your terminal (and assuming you’re still in the my-template-project folder), create a new folder alongside my-template-project, then run pulumi new to create a new project, pointing Pulumi to the folder containing your template:

$ cd ..
$ mkdir my-test-website && cd my-test-website
$ pulumi new ../my-template-project

Step through the prompts, which should include each of the configurable settings we defined in the template block earlier, along with their default values:

$ pulumi new ./my-template-project
project name: (my-test-website)
project description: (A simple static website on Amazon S3)
...
aws:region: The AWS region to deploy into: (us-west-2)
greeting: A greeting to show on the home page: (Hello, internet person! 👋) Hi, world!
title: A title to use for the home page: (My Awesome Website) Hey look, it's my website!
Your new project is ready to go! ✨
To perform an initial deployment, run `pulumi up`

Go ahead and deploy the new project with pulumi up, then navigate to the newly deployed website in your favorite web browser:

$ pulumi up --yes
$ open $(pulumi stack output url)

Well done! You’ve written your first Pulumi template.

Let’s finish things off by publishing your template so others can use it. As before, tidy up with pulumi destroy before moving on:

$ pulumi destroy --yes --remove

Step 5: Publish the template

A moment ago, when you tested your first template, you did so by pointing Pulumi to the template’s path on your local filesystem. Before that, when you ran pulumi new aws-yaml, you instructed Pulumi (implicitly) to fetch the source of the template over HTTPS from the official pulumi/templates repository on GitHub. So to make your template available to others, all you have to do publish it to a file path or Git URL that’s accessible by the Pulumi CLI.

You could create a new GitHub repository for this project, then pass the fully-qualified URL of the path containing Pulumi.yaml to pulumi new, like so:

$ pulumi new https://github.com/{your-org}/{your-repo}/tree/main

But since our project consists of just one file, you don’t even need to do that much: you can just paste the contents of the file into a new GitHub gist (since a gist is also a Git repository), name the gist Pulumi.yaml, and point Pulumi to the gist’s URL.

Go ahead and create a gist of your own, then use it to create a new project:

$ mkdir ../my-second-test-website && cd ../my-second-test-website
$ pulumi new https://gist.github.com/cnunciato/b331efae6a4740c237a0364d17fe220f
$ pulumi up

Be sure to tidy up as before with pulumi destroy when you’re done.

Bonus step: Add a Deploy with Pulumi button

In addition to the CLI, your users can also create new projects in the Pulumi Service with the Deploy with Pulumi button. This is a great option for making your project installable from GitHub READMEs and other team docs. Here, for example, is a Deploy button that creates a new project using my version of the gist we created above:

Embedding these buttons yourself is easy — just use one of the snippets below, swapping the values of {template-url} for the URL of your template’s gist or Git repository:

<a href="https://app.pulumi.com/new?template={template-url}">
 <img src="/images/deploy-with-pulumi/dark.svg" title="Deploy with Pulumi">
</a>

[![Deploy with Pulumi](/images/deploy-with-pulumi/dark.svg)](https://app.pulumi.com/new?template={template-url})

Project creators who go down this path will be prompted in the browser for the same configuration values as they would with the CLI, and afterward, they’ll be able to deploy the project either with the Pulumi CLI or with Pulumi Deployments.

See it in action

You can watch this Modern Infrastructure video to see another example of creating and sharing a Pulumi template:

Wrapping up

As you can see, creating a template is both simple and powerful, and I hope this post encourages you to experiment with a few of your own. Feel free to peruse the following links for inspiration:

And as always, be sure to stop by Pulumi Community Slack to let us know know how it goes.

Happy templating!

Announcing: Pulumi Deployments, YAML GA, Arch Templates

Luke Hoban — Wed, 02 Nov 2022 06:15:00 -0700

Some of the code in this post is out of date. See the AWS guides for an updated overview and examples.

Our mission at Pulumi is to enable teams to scale up what they can build in the cloud. Scale up the sophistication and value of their cloud infrastructure investments through software engineering practices. Scale up the automation around delivering cloud infrastructure with software instead of just humans. And scale up the number of developers who can directly benefit from the rich cloud platform capabilities being built by central platform teams in every organization today.

As part of Pulumi Cloud Engineering Days 2022 today we are announcing a set of important new advancements in the Pulumi platform which are all designed to help organizations scale with their infrastructure as code needs.

Pulumi Deployments: The Pulumi Service now supports executing infrastructure deployments as a managed service, driven by click-to-deploy in the Console, Git push-to-deploy from GitHub, programmatic deployments via the Pulumi Service REST API as well as automation via the Pulumi Automation API. Pulumi Deployments enables organizations to scale up their infrastructure automation.
Pulumi YAML GA: In addition to supporting rich general purpose programming languages like TypeScript, Python, Go, .NET and Java, Pulumi added support last May for Pulumi YAML as a simple, declarative interface for accessing the full breadth of the Pulumi Universal Infrastructure as Code platform. Pulumi YAML is now Generally Available as part of its 1.0 release, enabling everyone in an organization to use Pulumi to manage cloud infrastructure.
Architecture Templates: New templates providing architecture-specific starting points for the most common infrastructure as code use cases - Containerized Services, Kubernetes Clusters, Serverless Applications, Static Websites and Virtual Machines - available across all major cloud platforms and supported languages.
Project Level Configuration: As cloud infrastructure projects grow in complexity, Pulumi provides many tools to manage the complexity of the software that defines these projects. But the configuration of that software also grows in complexity. New support for Project-Level Configuration across all Pulumi languages makes it easier to manage complex configuration as cloud projects scale up.

You can check out the Cloud Engineering Days 2022 Keynote to learn more about all of these features, or get started with Pulumi today!

Pulumi Deployments

We launched the Pulumi Automation API two years ago to enable organizations to build their own automated systems using the Pulumi Infrastructure as Code engine. Over the last two years, we’ve seen incredible adoption of Automation API, with millions of infrastructure updates driven by Automation API, and usage in everything from CI systems, to CLI tools, to self-service provisioning platforms, to complex deployment orchestration software.

To scale up how we deploy and manage cloud infrastructure and ultimately unlock the cloud’s value and agility, we believe it will be critical to build software systems around our Infrastructure as Code platforms - systems that scale with software, not just humans.

Today, we are taking the next step in this journey with the introduction of Pulumi Deployments to the Pulumi Service. With Pulumi Deployments, you can run a Pulumi infrastructure as code action (a preview, update, destroy or refresh) inside Pulumi’s managed service. Pulumi provides scalability, observability and security for deployments.

Pulumi Deployments are exposed as a new fundamental capability of the service via the Pulumi Service REST API. They are also now supported in Automation API as a new kind of Remote workspace that can be used in building your own custom infrastructure automation software.

We’ve also used Pulumi Deployments to build new features in the Pulumi Service console, including the ability to click to deploy infrastructure or to set up Git push to deploy workflows that trigger previews on GitHub PR commits and updates on merges to a tracking branch. This provides a simple point-and-click interface to ensure that any Pulumi user can benefit from continuous delivery of infrastructure changes.

Of course, because Pulumi Deployments exposes a convenient programmatic API, you can build your own automation on top of this. In our Pulumi Deploy Demos, we have examples of building Drift Detection automation, Time-to-live self-destruct stacks, per-PR environments and more.

Pulumi Deployments is available in preview now, and you can signup to get access today. Pulumi Deployments is free while in preview. We can’t wait to see what you’ll build on top of Pulumi Deployments! 🚀

👉 Read more in the Pulumi Deployments blog post.

Pulumi YAML

We introduced Pulumi YAML last May as a simple declarative interface to the full breadth of the Pulumi platform, ideal for smaller scale use cases and composition of higher level component building blocks. Developers who may not have been ready to learn or adopt a full blown language for their cloud infrastructure can use YAML to get started.

Our goal has always been to offer the broadest range of programming language options to empower every cloud builder so that they could benefit from the best of Pulumi’s Universal Infrastructure as Code platform.

Pulumi YAML makes it easy to compose any of the tens-of-thousands of resources available to the Pulumi Universal Infrastructure as Code platform, including mixtures of raw cloud resources (like an AWS Lambda Function below) and higher level components authored in other rich Pulumi languages (like the aws-apigateway packages high level RestAPI below).

name: serverlessapp
runtime: yaml
config:
 roleArn:
 type: string
resources:
 fn:
 type: aws:lambda:Function
 properties:
 runtime: python3.9
 handler: handler.handler
 role: ${roleArn}
 code:
 fn::fileArchive: ./function
 api:
 type: aws-apigateway:RestAPI
 properties:
 routes:
 - path: /
 localPath: www
 - path: /date
 method: GET
 eventHandler: ${fn}
outputs:
 url: ${api.url}

Pulumi YAML is the only declarative Infrastructure as Code language that supports convert - the ability to convert a YAML program into a program in any other Pulumi language (TypeScript, Python, Go, .NET and Java). If the complexity of a project ever grows beyond what YAML can comfortably express, you can seamlessly adopt another richer language to continue to scale.

pulumi convert --language python

With the Pulumi YAML GA, we’ve made dozens of improvements to Pulumi YAML. Richer type checking and error reporting makes Pulumi YAML easy to get started with. And IDE integration for VS Code and Emacs makes Pulumi YAML even more productive to work with day-to-day:

👉 Read more in the Pulumi YAML GA blog post.

Architecture Templates

The new Pulumi Architecture Templates offer the easiest way to get started with a complete working piece of infrastructure for one of the most common Infrastructure as Code use cases. Each architecture template is short (~100 lines of code), builds a deployable infrastructure project out of the box, is easy to customize, uses a mix of raw cloud resources and Pulumi components to operate at the right level of abstraction for the use case, and comes with useful documentation.

The initial set of architecture templates support common infrastructure as code use cases across AWS, Azure, Google Cloud and Kubernetes, and in all supported Pulumi languages.

For example, the AWS Containerized Service template provides this starting point, which builds and deploys a local container-based application into a set of managed infrastructure in AWS using ECS, ALB, ECR and IAM resources:

from pulumi import Config, Output, export
import pulumi_aws as aws
import pulumi_awsx as awsx

config = Config()
container_port = config.get_int("containerPort", 80)
cpu = config.get_int("cpu", 512)
memory = config.get_int("memory", 128)

# An ECS cluster to deploy into
cluster = aws.ecs.Cluster("cluster")

# An ALB to serve the container endpoint to the internet
loadbalancer = awsx.lb.ApplicationLoadBalancer("loadbalancer")

# An ECR repository to store our application's container image
repo = awsx.ecr.Repository("repo")

# Build and publish our application's container image from ./app to the ECR repository
image = awsx.ecr.Image(
 "image",
 repository_url=repo.url,
 path="./app")

# Deploy an ECS Service on Fargate to host the application container
service = awsx.ecs.FargateService(
 "service",
 cluster=cluster.arn,
 task_definition_args=awsx.ecs.FargateServiceTaskDefinitionArgs(
 container=awsx.ecs.TaskDefinitionContainerDefinitionArgs(
 image=image.image_uri,
 cpu=cpu,
 memory=memory,
 essential=True,
 port_mappings=[awsx.ecs.TaskDefinitionPortMappingArgs(
 container_port=container_port,
 target_group=loadbalancer.default_target_group,
 )],
 ),
 ))

# The URL at which the container's HTTP endpoint will be available
export("url", Output.concat("http://", loadbalancer.load_balancer.dns_name))

Pulumi Architecture Templates build on Pulumi’s support for creating your own templates. Pulumi programs can be templated by users to create their own starting points for cloud infrastructure projects using pulumi new. You can build your own templates and Deploy with Pulumi buttons to create similar architecture templates within your own team and organization.

👉 Read more in the Pulumi Architecture Templates blog post.

Project Level Configuration

On top of support for defining infrastructure as code using traditional programming languages, Pulumi has always offered a rich configuration system for specifying inputs to a program which might differ between instances of that program (“stacks”) - for example, across the dev, staging and prod environments, or across regional deployments, of what is logically the “same” infrastructure definition. Configuration in Pulumi has so far only been provided at the stack level - each stack provides the complete set of configuration needed to specify how that instance of the Pulumi program will be parameterized.

With the latest Pulumi release, we are excited to be adding Project Level Configuration to Pulumi as well. With project-level configuration, you can specify the configuration needed for each stack once, inside the project file. This specification can offer default values across all stacks, types to enforce validation of values used in each stack, descriptions, and enforcement that certain properties must be secret.

name: baseinfra
runtime: python
config:
 vm-count:
 type: integer
 description: the number of virtual machines to create
 default: 2
 subnets:
 type: array
 description: an array of subnets to create
 items:
 type: string
 vaultKey:
 type: string
 description: which vault key to use for this stack
 secret: true

This is the first step in our broader goals to bring Hierarchical Configuration to Pulumi, avoiding unnecessary copy/paste, simple abstraction of configuration data, and rich validation and contextual user interface support across all aspects of config in the Pulumi experience.

👉 Read more in the Project Level Configuration blog post.

Conclusion

Pulumi helps cloud engineering teams scale up what they can accomplish in the cloud. Today, we announced a collection of new features and enhancements that advance that platform and help you scale your cloud infrastructure teams. These features give platform team new powers to scale up what they can accomplish.

Check out the launch blog posts for each of the new features, sign up for access to Pulumi Deployments, or just dive right in and get started with Pulumi today!

Pulumi YAML General Availability

Luke Hoban — Wed, 02 Nov 2022 06:05:00 -0700

Earlier this year we launched support for Pulumi YAML as a new supported language for Pulumi’s Universal Infrastructure as Code platform. Pulumi YAML offers a simple declarative interface to the full breadth of the Pulumi platform, ideal for smaller scale use cases and composition of higher level component building blocks. And with support for pulumi convert, Pulumi YAML programs can be converted into a program in any other Pulumi language, ensuring you can easily scale up if and when needed.

Today, we’re excited to announce the General Availability of Pulumi YAML with the release of Pulumi YAML 1.0.

GA includes many significant updates to the Pulumi YAML support introduced since the initial preview launch:

Pulumi YAML 1.0: The Pulumi YAML document format is now stable and will be kept backward compatible.
Full support for pulumi convert: Now supporting all Pulumi YAML features and all Pulumi languages.
IDE Support: IDE support for VS Code and Emacs, built on a Pulumi YAML LSP (Language Server Protocol) that can be used to light up improved tooling experience in any IDE.
Type Checking and Diagnostics: Significant improved error messages, and early validation of type-correctness, to provide a great developer experience.
Simplified function syntax: A simpler function syntax to more easily invoke Pulumi Package Functions.
New str package: A new Pulumi Package with many string manipulation functions that can be used in Pulumi YAML and other Pulumi languages.
Pulumi Kubernetes Operator Inline Program Support: Support for managing Pulumi cloud resources from directly within the Kubernetes resource model.

You can get started with Pulumi YAML today!

Why Pulumi YAML?

Cloud teams love YAML and markup languages due to the simplicity and ease of getting started. At Pulumi we work with organizations every day that have reached the “late-stage YAML endgame” - hundreds of thousands of lines of markup that have been copy/pasted. Small config changes like upgrading cluster versions require hunting down dozens or hundreds of files. Cloud teams end up feeling hamstrung, spending all of their time in maintenance and unable to invest in infrastructure for new products.

That is why we introduced Pulumi YAML - a simple markup frontend with the power to grow up with your organization. Pulumi YAML is the only infrastructure markup language that supports convert - the ability to automatically translate into any Pulumi Programming language. When you feel that your infrastructure complexity is reaching the endgame, you can pulumi convert to translate your infrastructure in full fidelity into Java, Go, Python, Typescript, or C# in order to tame complexity, and use the power of programming languages to encapsulate, design, and componentize your infrastructure.

Adopting Pulumi up until the introduction of Pulumi YAML required operations teams to learn a programming language such as Python or Javascript. We are yet to see a team that is unsuccessful in this transition, but many have asked for a simple markup language that they could use as an on-ramp. This allows them to start with tools that they know, and incrementally adopt the Pulumi ecosystem of productivity tools including Automation API, Pulumi Deploy, Multi-Language Components, and Policy as Code.

Pulumi YAML is also a powerful interchange format for programmatic tools. Since YAML can be easily interpreted by machines, it is the perfect tool to support custom CLIs, REST APIs and embedding within custom data formats. For example, the Pulumi Kubernetes Operator now uses Pulumi YAML to embed Pulumi programs within a pulumi.com/v1/Program CRD.

Critically, Pulumi YAML is designed to work with Pulumi’s CrossCode technology. You can write a complex component in Go, TypeScript or Python, and expose a simple user interface in YAML - realizing both the power and encapsulation of programming languages with the simplicity of a familiar markup language.

Pulumi YAML 1.0

Pulumi YAML offers a simple YAML document model for defining infrastructure using any of Pulumi’s more than 100 resource provider packages.

This includes 4 top level sections:

resources: A collection of resource definitions, each specifying a name, type, properties and options for a Pulumi resource definition.
outputs: A set of computed output values that will be available as Pulumi stack outputs.
config: The config that each instance of this project can set independent values for.
variables: Give names to complex values that might be needed in the program.

Here is an example Pulumi YAML document, from the AWS Serverless Application template.

name: serverlessapp
runtime: yaml
resources:
 role:
 type: aws:iam:Role
 properties:
 assumeRolePolicy:
 fn::toJSON:
 Version: 2012-10-17
 Statement:
 - Action: sts:AssumeRole
 Effect: Allow
 Principal:
 Service: lambda.amazonaws.com
 managedPolicyArns:
 - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
 fn:
 type: aws:lambda:Function
 properties:
 runtime: python3.9
 handler: handler.handler
 role: ${role.arn}
 code:
 fn::fileArchive: ./function
 api:
 type: aws-apigateway:RestAPI
 properties:
 routes:
 - path: /
 localPath: www
 - path: /date
 method: GET
 eventHandler: ${fn}
outputs:
 url: ${api.url}

Since preview, the config section has been introduced to replace the previous configuration section. This change was made to enable a standard config section in Pulumi.yaml across all Pulumi languages, to enable the new Project-level config. This section has mostly the same set of features as the preview configuration section, but uses more consistent naming for the Pulumi programming model:

config:
 numInstances:
 type: integer
 default: 3
 availabilityZones:
 type: array
 items:
 type: string
 default: [us-east-1-atl-1a, us-east-1-chi-1a]

Full support for `pulumi convert`

An important goal for Pulumi YAML is to be a safe starting point for developing infrastructure, with the ability to smoothly scale up into any other Pulumi language if or when needed. This ensures that using Pulumi YAML won’t limit how far you can scale up your infrastructure. If you need features that Pulumi YAML doesn’t support (control flow, multiple files, component authoring, etc.), or if the complexity of your YAML grows beyond what you want to manage in YAML, just pulumi convert and keep going in another language.

Pulumi Convert now handles all Pulumi languages, and all features of Pulumi YAML. For example, the original example program above can be converted to Python with:

pulumi convert --language python

This generates a Python program defining the exact same resources, along with all required dependencies. A pulumi up on the new program will report no changes, and you can now continue evolving the cloud resources managed by the stack using the new language.

pulumi convert is also useful for creating examples in all Pulumi languages. Authors of Pulumi Component Packages can write examples once in Pulumi YAML, and convert into other languages using pulumi convert. This makes it possible to quickly support great documentation for all Pulumi languages when building and publishing a new component.

IDE Support (Preview)

A critical value of using other Pulumi languages is the tight feedback loop developers get when working inside their IDE. We wanted to bring as much of this benefit as possible to Pulumi YAML as well.

With the new Pulumi YAML LSP (Language Server Protocol), developers can install an extension in their favorite IDE which offers many important IDE features for Pulumi YAML programs. Extensions for VS Code and Emacs are available now, and others will be released in the future. These support the following features:

Error squiggles: Highlight invalid keywords and property names in Pulumi YAML documents.
Hover tips: Hover over types to see rich inline documentation.
Autocomplete: Get hints and autocomplete on property names.

You can see what error squiggles (for a simple type) and hover tips look like in VS Code in this example:

Type Checking and Diagnostics

Good error messages make it significantly easier to quickly identify and fix issues in a Pulumi program. Pulumi YAML now checks programs against the resource schema of the resources being created, and reports early errors with rich information about the location of the error and suggestions on potential fixes.

For example, the same typo from the screenshot above leads to this helpful diagnostic message:

Simplified function syntax

Pulumi offers core support for functions defined in any Pulumi package, and Pulumi YAML programs can use these to be able to express everything available in the Pulumi package ecosystem.

It is now even easier to use these Pulumi functions via the fn intrinsics. There are two classes of functions available in Pulumi YAML:

Built-in Functions: Functions like fn::toJSON and fn::fromBase64 are built-in to Pulumi YAML, and enable common computations on values that come from configuration or other resource outputs.
Package Functions: Functions like fn::azure-native:storage:listStorageAccountKeys and fn::oci:ObjectStorage:getNamespace invoke a function from a Pulumi Package based on its package name, module and function name (fn::<package-name>:<module-name>:<function-name>).

New `str` Package

The new [str] package in the Pulumi Registry offers a suite of string manipulation functions which can be used from Pulumi YAML, as well as in other Pulumi languages. Functions supported so far:

str:replace: Replace occurrences of a substring with another substring.
str:trimSuffix: Trim a suffix from a string.
str:trimPrefix: Trim a prefix from a string.
str:regexp:split: Split a string on a regex.
str:regexp:replace: A regex based replace on a string.

These functions can all be invoked using the simple Package Functions syntax - for example, the following function call replaces bar with buzz in the given resource output:

prop:
 fn::str:replace:
 string: ${res.out}
 old: bar
 new: buzz

Pulumi Kubernetes Operator Inline `Program` Support

The Pulumi Kubernetes Operator recently added support for defining Pulumi resources directly inside Kubernetes YAML using a new pulumi.com/v1/Program CRD.

This support builds uniquely on top of Pulumi YAML, to provide a natural way to describe a set of Pulumi resources inline inside Kubernetes YAML using the Pulumi YAML document format embedded within the program key of a pulumi.com/v1/Program resource.

This allows the Kubernetes resource model to be used to define and evolve any of the tens-of-thousands of cloud resources supported by the Pulumi package ecosystem, and with access to the full breadth of IaC capabilities that Pulumi offers.

apiVersion: pulumi.com/v1
kind: Program
metadata:
 name: staticwebsite
program:
 resources:
 bucket:
 type: aws:s3:Bucket
 properties:
 website:
 indexDocument: index.html
 index.html:
 type: aws:s3:BucketObject
 properties:
 bucket: ${bucket.id}
 content: <h1>Hello, world!</h1>
 contentType: text/html
 acl: public-read
 outputs:
 url: http://${bucket.websiteEndpoint}
---
apiVersion: pulumi.com/v1
kind: Stack
metadata:
 name: staticwebsite
spec:
 stack: lukehoban/staticwebsite/test
 programRef:
 name: staticwebsite
 destroyOnFinalize: true
 config:
 aws:region: us-east-1

Conclusion

The GA of Pulumi YAML marks a stable foundation for the simple, declarative YAML interface to the Pulumi ecosystem. We’re excited to see YAML support expand the audiences who can jump in and use the best of the Pulumi ecosystem, ensuring that Pulumi offers a great experience for every Infrastructure as Code use case and user.

Check out these resources to get started with Pulumi YAML today:

Create an AWS Static Website Fast with Angular and Pulumi

Sean Holung — Wed, 24 Aug 2022 00:00:00 +0000

In this blog post, we’re going to use some Angular framework components to assemble a static website and then use Pulumi and its AWS Static Website component to deploy it to AWS. The website is for a café called the Pulumi Café. It will contain two pages, one an About page and the other a Menu page, as well as some navigational pieces.

To follow this example, you need to have both Angular and Pulumi installed. (Here’s a link to the Pulumi installation instructions.) You’ll also need an AWS account.

What is Angular?

Angular is an open-source UI framework for building single-page client applications with HTML and TypeScript. Angular itself is written in TypeScript and its architecture follows the Model-View-Controller (MVC) design pattern.

What are Components in Angular?

In Angular, the basic building blocks of a website are called components. These are reusable structures that contain their own logic and functionality. For example, you could create a button as an Angular component and program it to have a particular look and a particular behavior. Then, you would reuse the button component wherever you need it in the website. The button could also be configurable meaning that, based on different properties, you could change its look and behavior.

A simple website, like the one we’ll be building in this post, is mainly created from a group of Angular components.

What is the AWS Static Website Component?

The AWS Static Website component from Pulumi makes it easy to deploy an AWS S3 static website and, optionally, add a CloudFront content distribution network (CDN). While you can use any of the programming languages Pulumi supports (TypeScript, JavaScript, Python, Go, .NET, Java, and YAML), we’ll be using TypeScript, since that is what the Angular framework uses.

The great advantage of the AWS Static Website component is that you can have a complete, functioning site in a few minutes. The component encapsulates all the complex logic that an AWS static website requires. Without the component, you would need to spend hours or even days writing and debugging code to get the same result.

If you’re interested in learning more about the component, read Deploy Static Websites to AWS using 10 lines of YAML.

The Angular Components

All the Angular components you need to build the website are available in the GitHub repository. The repository contains:

A component for the left navigation bar.
A component for the top navigation bar.
A component for the Home page.
A component for the Menu page.

We also use some components that come with the Angular framework.

Each of the components has a TypeScript file that contains the programming logic and an HTML file that describes how the component will render.

Building the Pulumi Café Website

Here are the instructions to build the website.

Clone the GitHub repository located here.
Run npm install to install the website’s node dependencies.

Deploy the Site with Pulumi

You’re now ready to set up Pulumi to deploy the site to AWS.

Create a directory named pulumi in the root of the project directory and then navigate to it with this command:
```
mkdir pulumi && cd pulumi
```
You’re now in the pulumi directory. Generate a Pulumi project file and create a Pulumi.yaml file with the following command. You will be asked a few questions from the prompt. Use angular-static-website for the project name and accept the suggested defaults for the remaining questions:
```
pulumi new aws-yaml
```

Modify the .yaml File

We need to add a few lines of code to the Pulumi.yaml file. In the resources section, the code references the AWS Static Website component and sets the sitePath property to the location of the dist build directory. We also want to know the URL of the site once it’s deployed. That’s what the outputs section does. The websiteURL parameter is one of the outputs that the AWS Static Website component provides. Replace the contents of Pulumi.yaml with the following code:

name: angular-static-website
runtime: yaml
description: Example of deploying Angular with Pulumi
resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "../dist/"
outputs:
 websiteURL: ${web.websiteURL}

The AWS Static Website component is the reason you only need a few lines of code to deploy the site. By using it, all the work of setting up AWS is done for you.

To deploy the website, type pulumi up. You’ll first see a preview of all the resources that will be created. Confirm that you want to proceed and Pulumi deploys the website to AWS.

It’s that simple. With just a few lines of code and one command you deployed your site. Navigate to the site and you’ll see the Home page for the café.

You can see the navigation components on the left-hand side. Click Menu to see the Menu page.

Manage Your Site with Pulumi

Along with making it simple to deploy your site, Pulumi also makes it easy to perform some management tasks. For example, If you want to take down the site, run pulumi destroy. You never have to worry that you’ll forget to remove some resource and continue to pay for it. Pulumi also keeps track of changes to website files, can generate resource graphs and give you complete information about the resources you’re using.

Learn More

If you want to learn more about Pulumi and the AWS Static Website component, here are a few links you might find interesting:

Along with the Deploy Static Websites to AWS using 10 lines of YAML blog post, take a look at the AWS Static Website reference page.
If you want to learn about other Pulumi components, the registry page catalogs them all.
If you want to view more examples of how to use Pulumi, go to the Pulumi examples repo.
If you want to delve deeper into Pulumi, the Getting Started guide is a good first step.
If you’re interested in learning more about Pulumi concepts, try the Concepts page.
If you want to learn more about Angular, see their Getting Started docs.

Deploy Static Websites to AWS using 10 lines of YAML

Sean Holung — Thu, 07 Jul 2022 00:00:00 +0000

The AWS Static Website component makes it easy to deploy an AWS S3 static website and, optionally, add a CloudFront content distribution network (CDN). While you can use any of the programming languages Pulumi supports (TypeScript, JavaScript, Python, Go, .NET, Java, and YAML), the component is particularly useful if you use YAML or JSON. With the AWS Static Website component, you’ll have a complete, functioning site in a few minutes. Without it, you can spend hours or even days to get the same result.

The component is designed to be simple to use and accessible to a broad range of developers. The only prerequisites are a basic knowledge of YAML or JSON, an AWS account, and an understanding of what a static website is. Even if you’re not used to thinking about infrastructure as code (IaC), you can use the component and let it handle the complexities that make setting up a website so time consuming.

You will need to have Pulumi on your machine to use the component because you run the pulumi up command to deploy the website. There’s a link to the installation instructions at the end of this article. You’ll also need an AWS account.

What the Component Does

The component deploys a static website to AWS and, optionally, a CloudFront CDN. You simply add a few lines of code, set a few configuration parameters, run pulumi up and let the component do the rest. The component sets up:

An Amazon S3 bucket
An Amazon CloudFront CDN (optional)
- An Amazon Route 53 DNS record
- An Amazon Certificate Manager SSL/TLS certificate

Some Examples

Let’s look at two examples of how to use the component. We’re going to use YAML in this post, but JSON is also supported as well.

Create a Website

This YAML code creates a basic static website.

name: react-static-website
runtime: yaml
description: Example of deploying React with Pulumi
resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "../my-website/build"
outputs:
 websiteURL: ${web.websiteURL}

This is the JSON equivalent of the code shown above.

{
 "name": "react-static-website",
 "runtime": "yaml",
 "description": "Example of deploying React with Pulumi",
 "resources": {
 "web": {
 "type": "aws-static-website:index:Website",
 "properties": {
 "sitePath": "../my-website/build"
 }
 }
 },
 "outputs": {
 "websiteURL": "${web.websiteURL}"
 }
}

This example code deploys a static React website but you can use the component with any other static website framework, such as Hugo. The only parameter that would change is the output directory specified in the sitePath property.

The first two lines of this example give the project a name and tell Pulumi that the code is YAML. The next line is simply a description of the project.

Next you allocate a resource. This is a Pulumi term. Resources are the fundamental units that make up your cloud infrastructure and Pulumi provides many resources that are ready for you to use. Here, you only need one: aws-static-website:Website, which is the AWS Static Website component. The resource has an input property, sitePath, which is the root directory that contains the website’s contents. Of course, you can use any path you want.

Finally, you’ll probably want to know the URL of your new site once it’s deployed. That’s what the outputs line does. websiteURL is one of the outputs that the component provides.

Get Started

To get started, go to the folder that contains your static website project. Then, follow these steps.

Create a directory named infrastructure alongside your website project directory with this command:
```
mkdir infrastructure && cd infrastructure
```
You’re now in the infrastructure directory. Generate a Pulumi.yaml file with this command:
```
pulumi new aws-yaml
```
Here’s an example of what the final directory structure looks like.
```
| react-project-dir
├── my-website
| ├── build
│ ├── index.html
│ ├── main.css
├── infrastructure
│ ├── Pulumi.yaml
```
Note: Pulumi will also add some other files to the infrastructure directory, such as a file that contains information specific to your Pulumi stack.

Copy the resources section and the outputs section from the example and add it to the end of the Pulumi.yaml file.

resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "../my-website/build"
outputs:
 websiteURL: ${web.websiteURL}

Deploy the Website

To deploy the website, type pulumi up. You’ll first see a preview of all the resources that will be created. You then confirm that you want to proceed and Pulumi deploys the website.

What You Gain

The entire example has 10 lines of YAML code. With this, you will get an S3 bucket provisioned with all of your website’s contents. If you looked at the preview screen when you ran pulumi up, you saw the many resources that actually had to be deployed to create the site. You never see any of that complexity. Here’s an example of what’s happening under the hood.

If you follow the View Live link, you will go to the Pulumi Service, where you can see all of the information available about a deployment and explore the resources that were deployed. For example, you can see a resource graph and a list of resources. Here’s an example of a resource graph.

After Pulumi deploys the website, it will display any outputs you specified which, in our example, is the URL of the site. as well as the number of resources created.

Input Properties

The AWS Static Website component has several input properties. Here are three of the most commonly used.

sitePath - the root directory that contains the website’s contents (required).
withCDN - sets a flag to provision a CloudFront CDN.
targetDomain - the domain that serves the content. A Route53 hosted zone must exist for this domain if you specify this option.

See the component’s reference page for the other properties this component supports.

Output Properties

The AWS Static Website component has several output properties. The one you’ll probably use the most is websiteURL, which is the URL to access the website. See the component’s reference page for the others.

Add a CloudFront CDN

To add a CloudFront CDN, you only need to add one more line of code.

name: react-static-website
runtime: yaml
description: Example of deploying React with Pulumi
resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "./my-website/build"
 withCDN: true
outputs:
 websiteURL: ${web.websiteURL}

All you have to do is add the line withCDN: true and the component sets up the CDN instance for you. Again, run pulumi up, confirm that you want to proceed, and Pulumi deploys the website. Here’s an example of what you’ll see.

And here’s the output.

More Pulumi Advantages

We’ve already seen how much Pulumi simplifies creating a website. Pulumi also keeps track of any changes you make to any of the website files. It makes those changes the next time you run pulumi up. Here’s an example screenshot. The changes are marked in the Plan column.

If you simply want to take an inventory of what’s changed, you can run pulumi up and not proceed with the confirmation. Alternatively, you can run pulumi preview.

If you want to take down the site, run pulumi destroy. Pulumi marks all the resources for deletion. It first previews the changes and, if you confirm, removes them from your AWS account. You don’t have to worry that you’ll forget to remove some resource and continue to pay for it. Here’s an example of what you’d see if you ran pulumi destroy on your AWS S3 static website.

Learn More

If you’re curious and want to learn more about Pulumi and the AWS Static Website component, here are a few links you might find interesting:

You can examine the GitHub repository that contains the code for the component.
If you want to quickly try out the component, this GitHub repo houses a complete React example project that you can clone.
If you want to learn about other Pulumi components, take a look at the registry page, which catalogs them all.
If you want to view more examples of how to use Pulumi, take a look at the Pulumi examples repo.
If you want to delve deeper into Pulumi, the Getting Started guide is a good first step.
If you’re interested in learning more about Pulumi concepts, try the Concepts page.

Install Pulumi

To install Pulumi for AWS, go to Before You Begin. There are instructions for macOS, Windows and Linux. You need to have an AWS account.

Shared configuration stacks with AWS Systems Manager

Christian Nunciato — Thu, 30 Jun 2022 00:00:00 +0000

One thing I love about Pulumi is how easy it is to configure a stack. As a builder mainly of web applications, I’m always thinking about how I’ll configure my apps from one environment to the next, and being able to use Pulumi’s built-in support for configuration and secrets to manage the API keys and database credentials for my dev, staging, and production stacks individually is incredibly convenient.

For larger teams and organizations, though, where multiple applications rely on a set of common configuration settings — dozens of apps, say, depending on the same API service or database — having to keep all of those config settings in sync across all of those individually can become a bit of a pain. When this happens, you may find yourself looking for ways to extract those settings into some sort of a service to allow you to manage them easily in one place, and in a way that allows any application to inherit them automatically.

There are lots of ways of addressing this sort of problem. One that I like is AWS Systems Manager (SSM) — specifically, SSM Parameter Store. With Systems Manager and Parameter Store, you can store plain-text and encrypted secrets and expose them easily to any other resource in your AWS infrastructure. Parameter Store is a powerful tool to have in your cloud-programming toolbox — and combined with Pulumi, and in particular Pulumi stack references, you can use it to build specialized stacks make managing shared configuration simple.

In this post, I’ll show you one way to do this. We’ll start with a simple Pulumi stack that defines a few config settings with Parameter Store, then stand up a couple of other stacks to illustrate how to inherit and use those settings. We’ll also use Pulumi YAML, one of the newest additions to the Pulumi language family, to give you an opportunity to kick the tires on that as well.

Let’s get going.

Sketching it out

The plan is to begin with a Pulumi stack to act as the keeper of our hypothetical organization’s shared configuration. This stack will define two config settings: a message of the day, which we’ll store in plain text, and a secret message of the day (ssh!), which we’ll store as an encrypted secret. Later, we’ll build two stacks to act as downstream consumers of this shared-config stack — one, a static website that’ll use our config values to render some text in the browser, the other a serverless function that’ll return those values as JSON. Both will use Pulumi stack references to obtain their config values from Parameter Store. Here’s a visual:

Let’s start Stack 1 — the shared-config stack.

Building the shared config stack

First, make sure you’ve installed and configured Pulumi for AWS, and then, in your shell of choice, create a new folder to house our three stacks-to-be:

$ mkdir shared-config-with-ssm && cd shared-config-with-ssm
$ mkdir shared-config my-website my-service

This’ll leave you with three folders to work with, one for each stack:

$ tree .
.
├── my-service
├── my-website
└── shared-config

Change to the shared-config folder and generate a new Pulumi YAML project, accepting the defaults to create a new dev stack:

$ cd shared-config
$ pulumi new aws-yaml

When the new-project wizard completes, add two new settings for the dev stack with pulumi config set:

$ pulumi config set motd 'Hello from Pulumi!'
$ pulumi config set motd_secret 'Ssh! This is a secret.' --secret

These are the two values we’re going to make available to Stacks 2 and 3.

Now let’s build out the Pulumi program to provision these items as parameters in AWS Systems Manager. Replace the contents of Pulumi.yaml with the following YAML program, which reads the config values we just set, declares two new aws.ssm.Parameter resources (using fully-qualified paths as a best practice), and exports their parameter names as Pulumi stack outputs:

name: shared-config
runtime: yaml

# The values we're reading from stack configuration.
configuration:
 motd:
 type: String
 motd_secret:
 type: String

# The AWS SSM Parameters managed by this stack.
resources:
 motd_param:
 type: aws:ssm:Parameter
 properties:
 name: /shared-config/dev/motd
 value: ${motd}
 type: String
 motd_secret_param:
 type: aws:ssm:Parameter
 properties:
 name: /shared-config/dev/motd_secret
 value: ${motd_secret}
 type: SecureString

# Outputs are the fully-qualified AWS SSM Parameter names
# that consuming stacks can use to retrieve associated values.
outputs:
 motd_param_name: ${motd_param.name}
 motd_secret_param_name: ${motd_secret_param.name}

At this point, you might be wondering why we’re only exporting our parameters’ names, and not their values.

We certainly could export their values, since we already have them at hand. But if we did, there wouldn’t be much point to using Parameter Store at all, since Pulumi itself is perfectly capable of managing plain-text and encrypted values, and stack references give other stacks an easy way of accessing them programmatically. Why not export the values, then?

For one, it’d be less flexible. If we exposed these values as stack outputs, downstream stacks referencing them would only be able to read them at deploy-time — i.e., during their own Pulumi updates. For shared-config values that change rarely, that might be okay, but for others, it’d be less than ideal in that all downstream stacks would have to be updated immediately whenever one of those values were changed.

Secondly, it keeps secrets a bit more secret. Pulumi is great at protecting the secrets it manages, even across stack boundaries like these, but applications still need a way to get access to the underlying plain-text values of secrets at runtime. Lambda functions, for example, need access tokens and database passwords to do the work we need them to do — but figuring out how to provide them with those raw values safely can be tricky. By requiring client applications to fetch secrets from Parameter Store, we can keep their plain-text values out of the bodies of our Lambda functions and prevent them from surfacing inadvertently in the AWS Lambda Console.

Finally, it leaves room for the occasional out-of-band change. Now and then, you might be faced with having to update a value in Parameter Store directly — e.g., with the AWS Console or CLI. Downstream stacks not reading from Parameter Store, however, wouldn’t have access to these updated values until after the shared-config stack were refreshed and redeployed — and would themselves have to be redeployed in order to use them.

Perhaps that was more explanation than necessary for what amounts to just a few lines of YAML, but hopefully it gives you a sense of the kinds of things to be thinking about as you explore solutions like this on your own. For now, let’s forge ahead and get this first stack deployed.

Back in your terminal, do that with pulumi up. The deployment should take only a few seconds to complete:

$ pulumi up
...

Updating (dev)

 Type Name Status
 + pulumi:pulumi:Stack shared-config-dev created
 + ├─ aws:ssm:Parameter motd_param created
 + └─ aws:ssm:Parameter motd_secret_param created

Outputs:
 motd_param_name : "/shared-config/dev/motd"
 motd_secret_param_name: "/shared-config/dev/motd_secret"

Resources:
 + 3 created

Duration: 4s

Notice we’ve named our outputs clearly, and that the secret message is safely encrypted in Pulumi.dev.yaml and in Parameter Store. You can confirm the latter by visiting the AWS Console:

Now let’s put these two new values to practical use.

Fetching configuration at deploy-time

Assuming you’re still in the shared-config folder, change to the my-website folder you created a few moments ago and generate another YAML project, this one for our humble little website:

$ cd ../my-website
$ pulumi new aws-yaml

Once again, follow the prompts to create a new dev stack, then replace the contents of Pulumi.yaml with the program below. Here, we’re fetching the names of the parameters being managed by shared-config, pulling their values from Systems Manager, and creating a one-page website to render those values in the browser. (Be sure to adjust the pulumi:pulumi:StackReferences resource to point to your stack instead of mine.)

name: my-website
runtime: yaml

variables:

 # Get the names of the parameters we care about from the shared-config stack.
 motd_param_ref: ${my-stack-reference.outputs["motd_param_name"]}
 motd_secret_param_ref: ${my-stack-reference.outputs["motd_secret_param_name"]}

 # Fetch (and decrypt) their values from Systems Manager.
 motd_param:
 Fn::Invoke:
 Function: aws:ssm:getParameter
 Arguments:
 name: ${motd_param_ref}
 motd_secret_param:
 Fn::Invoke:
 Function: aws:ssm:getParameter
 Arguments:
 name: ${motd_secret_param_ref}
 withDecryption: true

resources:
 # Create the stack reference
 my-stack-reference:
 type: pulumi:pulumi:StackReference
 properties:
 name: cnunciato/shared-config/dev  # <-- Change this.

 # Create an S3 bucket and configure it as a website.
 my-bucket:
 type: aws:s3:Bucket
 properties:
 website:
 indexDocument: index.html

 # Create a homepage and render the values.
 index.html:
 type: aws:s3:BucketObject
 properties:
 bucket: ${my-bucket}
 acl: public-read
 contentType: text/html
 content: |
 <html>
 <head>
 <meta charset="utf-8">
 <style>
 span { display: none; }
 button:active + span { display: inline; }
 </style>
 </head>
 <body>
 <h1>${motd_param.value}</h1>
 <button>Reveal the secret message!</button>
 <span>${motd_secret_param.value}</span>
 </body>
 </html>

outputs:

 # Export the publicly accessible URL of the website.
 url: http://${my-bucket.websiteEndpoint}

Save the file and deploy the stack, which again should take only a few seconds to complete:

$ pulumi up
...

Updating (dev)

 Type Name Status
 + pulumi:pulumi:Stack my-website-dev created
 + ├─ aws:s3:Bucket my-bucket created
 + └─ aws:s3:BucketObject index.html created

Outputs:
 url: "http://my-bucket-269e711.s3-website-us-west-2.amazonaws.com"

Resources:
 + 3 created

Duration: 5s

With the exported URL, open the website in your favorite browser, and you should also see the message of the day on the homepage, along with a button you can click to reveal the secret message:

If you’re using the Pulumi Service, which tracks dependencies between stacks, you should see that the my-website stack is now listed as a downstream consumer of shared-config:

Now, when the hypothetical team managing the shared-config stack decides it’s time to update the message of the day, it can do so with a pulumi config set and pulumi up, and my-website will be able to pick up the new message automatically.

However, it’s worth mentioning that that won’t happen immediately, as the my-website stack is configured to pull the message from Systems Manager at deploy-time. If we want to reflect the new value on the homepage, we’ll either need to deploy the my-website again or come up with a way to fetch the message at runtime — which brings us to our third and final stack, in which you’ll learn how to do just that.

Fetching configuration at runtime

The easiest way to see this in action is with AWS Lambda. With Lambda and Pulumi’s support for function serialization, we can capture the parameter names with stack references, pass them into the Lambda at deploy-time (as plain-text strings, which is fine, because they’re just names), and fetch their values from Systems Manager at runtime — i.e., when the Lambda is invoked sometime later — with the AWS SDK for Node.js. This way, any updates made by the shared-config team will be usable by our Lambda function immediately, no redeployment necessary.

So to finish things off, change to the my-service folder you created earlier and generates a third and final project, this one with TypeScript, again accepting the defaults to create a dev stack for it:

$ cd ../my-service
$ pulumi new aws-typescript

Replace the contents of index.ts with the following program. As before, we’re using StackReferences to read our parameter names from the shared-config stack — except here, because StackRerefences are provided as pulumi.Outputs, which can’t be serialized into the Lambda, we’ll need to use .get() to unwrap the parameter name into a string so we can use it at runtime with the AWS SDK:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Get the names of the parameters we care about from the shared-config stack.
const stack = new pulumi.StackReference("cnunciato/shared-config/dev"); // <-- Your name here.
const motdParamRef = stack.requireOutput("motd_param_name");
const motdSecretParamRef = stack.requireOutput("motd_secret_param_name");

const callback = new aws.lambda.CallbackFunction("lambda", {
 policies: [
 aws.iam.ManagedPolicy.LambdaFullAccess,

 // Give the Lambda read-only access to AWS Systems Manager.
 aws.iam.ManagedPolicy.AmazonSSMReadOnlyAccess
 ],
 callback: async () => {

 // Use the AWS SDK for JavaScript to fetch parameter values from Systems Manager.
 const ssm = new aws.sdk.SSM();

 // Fetch the message of the day.
 const motdParam = await ssm.getParameter({
 Name: motdParamRef.get(), // <-- Use .get() for you runtime access to Output values.
 }).promise();

 // Fetch the secret message of the day.
 const motdSecretParam = await ssm.getParameter({
 Name: motdSecretParamRef.get(),
 WithDecryption: true,
 }).promise();

 return {
 statusCode: 200,
 contentType: "application/json",

 // Return the result.
 body: JSON.stringify({
 motd: motdParam.Parameter?.Value,
 motd_secret: motdSecretParam.Parameter?.Value
 }),
 };
 },
});

const lambdaUrl = new aws.lambda.FunctionUrl("lambda-url", {
 functionName: callback.name,
 authorizationType: "NONE",
});

// Export the publicly accessible URL of the service.
export const url = lambdaUrl.functionUrl;

When you’re ready, deploy the my-service stack with pulumi up:

$ pulumi up
...

Updating (dev)

 Type Name Status
 + pulumi:pulumi:Stack my-service-dev created
 + ├─ aws:iam:Role lambda created
 + ├─ aws:lambda:Function lambda created
 + ├─ aws:iam:RolePolicyAttachment lambda-b63d666c created
 + ├─ aws:iam:RolePolicyAttachment lambda-b5aeb6b6 created
 + └─ aws:lambda:FunctionUrl lambda-url created

Outputs:
 url: "https://6y3vkpktyybylhua266bu7yplu0dgqgv.lambda-url.us-west-2.on.aws/"

Resources:
 + 6 created

Duration: 19s

Verify the deployment with a request to the Lambda function’s public endpoint:

$ curl $(pulumi stack output url)
{
 "motd": "Hello from Pulumi!",
 "motd_secret": "Ssh! This is a secret."
}

That tells you the runtime lookup is working, which is great.

Now try updating the message of the day. Change to the shared-config folder, use pulumi config set to update the stack’s configured value, and finally, run pulumi up to write the new value to Systems Manager:

$ cd ../shared-config
$ pulumi config set motd 'Hello *again* from Pulumi!'
$ pulumi up
...

Updating (dev)

 Type Name Plan Info
 pulumi:pulumi:Stack shared-config-dev
 ~ └─ aws:ssm:Parameter motd_param updated [diff: ~value]

Resources:
 ~ 1 updated
 2 unchanged

If all goes well, you should be able to invoke the Lambda again and see the new value immediately:

$ cd ../my-service
$ curl $(pulumi stack output url)
{
 "motd": "Hello *again* from Pulumi!",
 "motd_secret": "Ssh! This is a secret."
}

And with that, we’re done. You can tear everything down with pulumi destroy:

$ cd ..
$ pulumi -C my-service destroy
$ pulumi -C my-website destroy
$ pulumi -C shared-config destroy

Wrapping up, and next steps

Managing configuration at scale is a much bigger topic than we could possibly cover in a blog post, but hopefully you’ve learned enough here to fuel some experimentation of your own. You’ll find the full source code for this walkthrough on GitHub as well.

Additionally, as an alternative to Parameter Store, you might also want to check out AWS Secrets Manager. It’s built specifically for managing secrets, and gives you additional capabilities like rotation and auditing, so depending on your use case, it could be a better fit for certain scenarios.

Happy configuring!

Pulumi Universal IaC: New Support For Java, YAML and AWS CDK

Luke Hoban — Wed, 04 May 2022 07:00:00 -0700

Some of the code in this post is out of date. See the AWS guides for an updated overview and examples.

Over the last year since the launch of Pulumi 3.0, we’ve seen incredible growth in adoption and usage of the Pulumi open source project and Cloud Engineering platform, with more than a thousand new open source contributors, tens of thousands of new users, and millions of new cloud infrastructure resources being managed by Pulumi. Pulumi’s infrastructure as code tools are enabling teams to scale up their cloud infrastructure with robust software engineering tools and practices to get the most value out of their cloud platform investments.

Today, we’re excited to announce a wave of innovation across the Pulumi project with a collection of significant new feature launches. These new features bring together Pulumi’s Universal Infrastructure as Code offering, supporting the widest range of builders, clouds, programming languages, and cloud architectures.

Today’s launches include:

Pulumi support for Java: New support for Java across the entire Pulumi platform, providing full access to the rich software engineering ecosystem for Java and the JVM.
Pulumi YAML: A new simple YAML interface to the Pulumi platform, broadening the reach of Pulumi to use cases where a full blown programming language is not (yet!) necessary.
Crosswalk for AWS in all Pulumi Languages: Support for the popular Crosswalk for AWS packages (for ECS, VPC, API Gateway and EKS) in all Pulumi languages.
AWS CDK on Pulumi: Support for deploying AWS CDK constructs from within Pulumi infrastructure as code programs, enabling access to the broad array of higher-level components for AWS built by teams at AWS and the AWS CDK community.
New Cloud and SaaS Providers: More than a dozen new packages in the Pulumi Registry for popular cloud/SaaS platforms from Pulumi partners, including Oracle Cloud, Elastic Cloud and Databricks.
Pulumi Service Provider for Pulumi: A new provider for managing the Pulumi Service using Pulumi Infrastructure as Code.
Pulumi CrossCode: Key improvements in the core technology powering Pulumi’s universal infrastructure as code features, including pulumi import, pulumi convert and more.

These new features build on the Pulumi 3.0 foundation to offer a truly universal infrastructure as code platform for modern cloud engineers and teams.

Pulumi Support For Java

Java is one of the most used programming languages in the world, with heavy usage in enterprises and organizations building for the cloud. The Java ecosystem has played a leading role in offering an incredibly rich ecosystem of software engineering capabilities - IDE tooling, strong typing, design patterns, build and package management, unit testing, and more.

Pulumi now supports Java (in addition to existing support for TypeScript, Python, Go and .NET) and other languages targeting the JVM. All the existing Pulumi CLI workflows can now be used with Java, and all packages in the Pulumi Registry are now available as Java libraries in Maven Central.

You can get started with Pulumi using Java today in the Pulumi Getting Started guide and the Pulumi and Java language docs.

👉 Read more in the Pulumi and Java blog post.

Pulumi YAML

Pulumi has differentiated itself as an Infrastructure as Code tool that embraces popular programming languages and their rich software engineering ecosystems. However, our goal has always been to offer the broadest range of language support for the wide variety of use cases and users who need to interact with cloud infrastructure in modern cloud organizations.

Pulumi YAML offers a new, simple interface to the full richness of the Pulumi Platform. Pulumi YAML allows infrastructure as code to be authored in a single simple YAML file with just a few lines of YAML, immediately familiar to any engineer, operator or developer.

name: yamldemo
runtime: yaml
resources:
 bucket:
 type: aws:s3:Bucket
 properties:
 website:
 indexDocument: index.html
 index.html:
 type: aws:s3:BucketObject
 properties:
 bucket: ${bucket.id}
 content: <h1>Hello, world!</h1>
 contentType: text/html
 acl: public-read
outputs:
 bucketName: ${bucket.id}

Like all other Pulumi languages, Pulumi YAML programs have access to all the core features of Pulumi’s infrastructure as code tooling, including native providers, secrets management, stack references, Pulumi Packages, and all the features of the Pulumi Service. YAML programs can interoperate seamlessly with the rest of the Pulumi ecosystem, consuming the outputs of other Pulumi programs and using Pulumi components built in existing Pulumi languages. For example, a complete EKS cluster can be configured using the EKS package (implemented in TypeScript):

name: eks-cluster
runtime: yaml
resources:
 cluster:
 type: eks:Cluster
outputs:
 kubeconfig: ${cluster.kubeconfig}

Access to Pulumi packages authored in other Pulumi languages means complexity can be pushed into reusable infrastructure components instead of copy-pasted in YAML. And when complexity does increase, Pulumi YAML supports a new pulumi convert command to eject into any other Pulumi programming language to manage the increased complexity via rich software engineering features in those languages.

You can get started with Pulumi YAML today in the Pulumi Getting Started guide and the Pulumi YAML language docs.

👉 Read more in the Pulumi YAML blog post.

Crosswalk for AWS in All Pulumi Languages

Crosswalk for AWS is a collection of libraries that use automatic well-architected best practices to make common infrastructure as code tasks in AWS easier and more secure. This includes libraries for EKS and API Gateway as well as the AWSX library that provides VPC, ECS, LoadBalancing and more. All of these libraries are now available in the Pulumi Registry for all Pulumi languages. The AWSX library, which was previously only available for Pulumi TypeScript users has become one of the most used libraries in the Pulumi ecosystem. With today’s release, it is now available to the entirety of the Pulumi language ecosystem - including Java and YAML!

For example - to run a Docker container in ECS using default network and cluster settings, users can use the awsx.ecs.FargateService class along with an Application Load Balancer listening on port 80 at a stable address - with just a few lines of code:

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

const cluster = new aws.ecs.Cluster("default-cluster");

const lb = new awsx.lb.ApplicationLoadBalancer("nginx-lb");

const service = new awsx.ecs.FargateService("my-service", {
 cluster: cluster.arn,
 desiredCount: 2,
 taskDefinitionArgs: {
 container: {
 image: "nginx:latest",
 cpu: 512,
 memory: 128,
 essential: true,
 portMappings: [
 {
 containerPort: 80,
 targetGroup: lb.defaultTargetGroup,
 },
 ],
 },
 },
});

export const url = lb.loadBalancer.dnsName;

import pulumi
import pulumi_aws as aws
import pulumi_awsx as awsx

cluster = aws.ecs.Cluster("default-cluster")

lb = awsx.lb.ApplicationLoadBalancer("nginx-lb")

service = awsx.ecs.FargateService("my-service",
 cluster=cluster.arn,
 desired_count=2,
 task_definition_args=awsx.ecs.FargateServiceTaskDefinitionArgs(
 container=awsx.ecs.TaskDefinitionContainerDefinitionArgs(
 image="nginx:latest",
 cpu=512,
 memory=128,
 essential=True,
 port_mappings=[awsx.ecs.TaskDefinitionPortMappingArgs(
 target_group=lb.default_target_group
 )],
 )
 )
)

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v5/go/aws/ecs"
 "github.com/pulumi/pulumi-aws/sdk/v5/go/aws/lb"
 ecsx "github.com/pulumi/pulumi-awsx/sdk/go/awsx/ecs"
 lbx "github.com/pulumi/pulumi-awsx/sdk/go/awsx/lb"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {

 cluster, err := ecs.NewCluster(ctx, "default-cluster", &ecs.ClusterArgs{})
 if err != nil {
 return err
 }

 loadBalancer, err := lbx.NewApplicationLoadBalancer(ctx, "nginx-lb", &lbx.ApplicationLoadBalancerArgs{})
 if err != nil {
 return err
 }

 _, err = ecsx.NewFargateService(ctx, "my-service", &ecsx.FargateServiceArgs{
 Cluster: cluster.Arn,
 DesiredCount: pulumi.Int(2),
 TaskDefinitionArgs: &ecsx.FargateServiceTaskDefinitionArgs{
 Container: &ecsx.TaskDefinitionContainerDefinitionArgs{
 Image: pulumi.String("nginx:latest"),
 Cpu: pulumi.Int(512),
 Memory: pulumi.Int(128),
 Essential: pulumi.Bool(true),
 PortMappings: ecsx.TaskDefinitionPortMappingArray{
 ecsx.TaskDefinitionPortMappingArgs{
 ContainerPort: pulumi.Int(80),
 TargetGroup: loadBalancer.DefaultTargetGroup,
 },
 },
 },
 },
 })
 if err != nil {
 return err
 }

 ctx.Export("url", loadBalancer.LoadBalancer.ApplyT(func(alb *lb.LoadBalancer) interface{} {
 return alb.DnsName
 }))
 return nil
 })
}

var cluster = new Aws.Ecs.Cluster("default-cluster");
var lb = new Awsx.Lb.ApplicationLoadBalancer("nginx-lb");
var service = new Awsx.Ecs.FargateService("my-service", new Awsx.Ecs.FargateServiceArgs
{
 Cluster = cluster.Arn,
 DesiredCount = 2,
 TaskDefinitionArgs = new Awsx.Ecs.Inputs.FargateServiceTaskDefinitionArgs
 {
 Container = new Awsx.Ecs.Inputs.TaskDefinitionContainerDefinitionArgs
 {
 Image = "nginx:latest",
 Cpu = 512,
 Memory = 128,
 Essential = true,
 PortMappings = {new Awsx.Ecs.Inputs.TaskDefinitionPortMappingArgs
 {
 TargetGroup = lb.DefaultTargetGroup,
 }},
 }
 }
});
this.Url = lb.LoadBalancer.Apply(lb => lb.DnsName);

name: awsx
runtime: yaml
resources:
 cluster:
 type: aws:ecs:Cluster
 lb:
 type: awsx:lb:ApplicationLoadBalancer
 nginx:
 type: awsx:ecs:FargateService
 properties:
 cluster: ${cluster.arn}
 taskDefinitionArgs:
 container:
 image: nginx:latest
 cpu: 512
 memory: 128
 portMappings:
 - containerPort: 80
 targetGroup: ${lb.defaultTargetGroup}
outputs:
 url: ${lb.loadBalancer.dnsName}

You can get started with Crosswalk for AWS in the Pulumi Crosswalk for AWS Guides, now available with examples in all languages.

👉 Read more in the Pulumi Crosswalk for AWS support for all Pulumi Languages blog post.

AWS CDK on Pulumi

The AWS Cloud Development Kit (CDK) offers a large collection of higher-level libraries (“constructs”) for working with the AWS platform, built by service teams at AWS and by the AWS CDK community. These libraries are available in the same set of general purpose programming languages that Pulumi supports, with the primary difference being that AWS CDK compiles infrastructure programs into CloudFormation, and uses CloudFormation to drive deployments.

With the new AWS CDK on Pulumi project, available in public preview today, we are opening up the ability to use AWS CDK constructs from within a Pulumi deployment. For developers already using AWS CDK, this provides Pulumi as a new option for orchestrating deployments in place of CloudFormation, offering improved deployment speed, integration with the full set of features of the Pulumi Cloud Engineering Platform (like Policy as Code, Audit Logs, Secrets, and much more). But even better, you can also now combine AWS CDK and Pulumi resources in a single Pulumi infrastructure as code project - passing outputs from Pulumi resources into AWS CDK constructs, and outputs from AWS CDK constructs into Pulumi resources. This allows you to work across the >80 cloud and SaaS providers that Pulumi offers access to, while still benefiting from high level libraries from the AWS CDK project.

import * as fs from 'fs';
import * as aws_events from 'aws-cdk-lib/aws-events';
import * as aws_events_targets from 'aws-cdk-lib/aws-events-targets';
import * as aws_lambda from 'aws-cdk-lib/aws-lambda';
import { CfnOutput, Duration } from 'aws-cdk-lib';
import * as pulumi from '@pulumi/pulumi';
import * as pulumicdk from '@pulumi/cdk';
import { Construct } from 'constructs';
import { remapCloudControlResource } from './adapter';

// Use Pulumi to construct an inline Lambda function
const lambdaFn = new aws.lambda.CallbackFunction('lambda', {
 callback: (ev, ctx) => console.log(ev),
});

class LambdaStack extends pulumicdk.Stack {
 lambdaArn: pulumi.Output<string>;

 constructor(id: string, options?: pulumicdk.StackOptions) {
 super(id, { ...options, remapCloudControlResource });

 // Use the AWS CDK Rule API directly.
 const rule = new aws_events.Rule(this, 'rule', {
 // Run 6:00 PM UTC every Monday through Friday
 schedule: aws_events.Schedule.expression('cron(0 18 ? * MON-FRI *)'),
 });

 const fn = aws_lambda.Function.fromFunctionArn(this, 'lambdaFn', pulumicdk.asString(lambdaFn.arn))

 // Use the AWS CDK to add a Rule target to trigger the Function.
 rule.addTarget(new aws_events_targets.LambdaFunction(lambdaFn));

 // Export the Lambda function's ARN as an output.
 this.lambdaArn = this.asOutput(lambdaFn.functionArn);

 this.synth();
 }
}

// Construct the CDK stack as a Pulumi component.
const stack = new LambdaStack('teststack');

// Use an output from the CDK construct from Pulumi.
export const lambdaArn = stack.lambdaArn;

This can be deployed with the pulumi CLI just like any other Pulumi program.

> pulumi up
Updating (dev)
View Live: https://app.pulumi.com/lukehoban/cdk-cron-lambda/dev/updates/37
Type Name Status
+ pulumi:pulumi:Stack cdk-cron-lambda-dev created
+ └─ cdk:index:Stack teststack created
+ └─ cdk:construct:LambdaStack teststack/teststack created
+ ├─ cdk:construct:Function teststack/teststack/lambda created
+ │ ├─ cdk:construct:Role teststack/teststack/lambda/ServiceRole created
+ │ │ └─ aws-native:iam:Role lambdaServiceRole494E4CA6 created
+ │ └─ aws-native:lambda:Function lambda8B5974B5 created
+ └─ cdk:construct:Rule teststack/teststack/rule created
+ ├─ aws:cloudwatch:EventRule ruleF2C1DCDC created
+ ├─ aws:cloudwatch:EventTarget Target0 created
+ └─ aws:lambda:Permission ruleAllowEventRuleteststacklambda47768855D6EFF36B created
Resources:
+ 11 created
Duration: 48s

AWS CDK on Pulumi is available for TypeScript today in public preview.

👉 Read more in the AWS CDK on Pulumi blog post.

New Cloud and SaaS Providers

Since launching the Pulumi Registry last fall, we’ve added more than a dozen new packages, supporting a variety of new cloud and SaaS providers.

Oracle Cloud is the most recent major new cloud provider, offering access to the entire surface area of Oracle’s Cloud platform. In addition, new providers for Elastic Cloud, DataBricks, jFrog Artifactory, Event Store Cloud, Scaleway, Slack, TailScale, Twingate, Checkly, Buildkite and more continue to expand the long tail of cloud and SaaS platforms accessible from Pulumi.

Every package in the Pulumi Registry supports all Pulumi programming languages, including Java and YAML. Documentation and getting started guides for every package are available from the Registry. Contributions to the Registry are now simpler than ever, with enhanced guides for authoring and publishing your own package in the Pulumi Registry.

New Provider for the Pulumi Service

Pulumi’s users love managing all of their cloud infrastructure using Pulumi, and that includes managing the state of the Pulumi Service itself.

Today, we released a new Pulumi provider for the Pulumi Service, supporting configuration of Teams, Access Tokens, Stack Tags and Webhooks using infrastructure as code.

For example, you can create a webhook that is notified whenever a Pulumi Update completes, and log it using an AWS Lambda, with just a few lines of code using the new Pulumi Service provider along with the API Gateway package.

import * as awsx from "@pulumi/awsx";
import * as pulumiservice from "@pulumi/pulumi-service";

const api = new awsx.apigateway.API("api", {
 routes: [{ path: "/", method: "POST", eventHandler: async (ev) => {
 const body = Buffer.from(ev.body!, 'base64').toString("ascii")
 console.log(JSON.stringify(body));
 return { statusCode: 200, body: "" };
 }}],
});

const webhook = new pulumiservice.Webhook("webhook", {
 active: true,
 organizationName: "upstarts",
 displayName: "pulumipulumi",
 payloadUrl: api.url,
});

👉 Read more in the Pulumi Provider for the Pulumi Service blog post.

Pulumi CrossCode

Pulumi CrossCode is the underlying set of technologies that Pulumi uses to support a wide range of programming languages, to enable sharing components and packages between languages, to support converting infrastructure from existing formats into Pulumi, to import infrastructure directly from cloud providers into Pulumi code, and to support converting Pulumi YAML into other Pulumi languages.

Support for Java and YAML was built on (and into) CrossCode to enable these languages to leverage everything that exists in the Pulumi ecosystem and to easily plug into existing features like conversion tools, example generation and so much more value that exists in the Pulumi platform.

In addition, recent improvements to the pulumi import command were enabled by CrossCode - making it possible to generate Pulumi Programs from existing cloud infrastructure.

And of course, the new pulumi convert command is powered by the existing features of CrossCode, opening up new paths for adoption and scaling of Pulumi projects.

Learn more about the technology underlying CrossCode in GitHub.

Summary

Pulumi’s Universal Infrastructure as Code platform supports the widest range of builders, clouds, programming languages, and cloud architectures available today.

With support for a broad range of languages - from the enterprise software engineering features of Java, to the simple declarative interface of Pulumi YAML - Pulumi scales from the simplest cloud use cases up into the most complex cloud infrastructure being managed today, and bridges the gap between cloud platform teams, operations teams, and application development teams.

With support for more than 80 cloud and SaaS providers, Pulumi can be used across multi-cloud workloads, and to coordinate the entire range of cloud infrastructure and managed services that define your cloud application - not just a single cloud provider. And with high level components for AWS from both Pulumi Crosswalk for AWS and AWS CDK on Pulumi, developers working in AWS have the richest set of high level infrastructure building blocks available for productively composing well-architected cloud infrastructure on AWS.

To learn more about each of the new features - check out these blog posts with more details:

We’re incredibly excited about today’s launches, but also see an amazing opportunity ahead to continue to drive new innovation in Infrastructure as Code and the future of Cloud Engineering. Can’t wait to see what you build!

Pulumi YAML: A Simple Declarative Interface for Pulumi

Luke Hoban — Wed, 04 May 2022 06:58:00 -0700

Since we first launched Pulumi 4 years ago, a core point of differentiation between Pulumi and other Infrastructure as Code offerings has been the ability to use popular general purpose programming languages - and their rich software engineering ecosystems - in order to scale up the complexity and richness of cloud infrastructure workloads. This approach has enabled cloud builders to adopt and embrace modern Infrastructure as Code with Pulumi using a wide variety of languages, including TypeScript, Python, Go, C# and Java.

Our goal though has always been to offer the broadest range of programming language options to empower every cloud builder so that they could benefit from the best of Pulumi’s Infrastructure as Code platform.

Today, we are excited to launch Pulumi YAML, a simple YAML-based interface to the entirety of the Pulumi Infrastructure as Code platform.

Pulumi YAML is a great option for smaller scale cloud infrastructure use cases. This includes:

Teams that don’t (yet) need the richness of a general purpose programming language, and feel more comfortable with YAML.
Simple infrastructure definitions which only need to compose a small number of cloud building blocks together.
Providing a simple interface to a complex set of high-level cloud components built with existing general-purpose Pulumi languages.
Systems which will generate infrastructure as code definitions programmatically – such as self-service provisioning portals.

Like all other Pulumi languages, Pulumi YAML programs have access to all the core features of Pulumi’s infrastructure-as-code tooling, including native providers, secrets management, stack references, Pulumi Packages, and all the features of the Pulumi Service. Critically, Pulumi YAML programs can interoperate seamlessly with the rest of the Pulumi ecosystem, consuming the outputs of other Pulumi programs and using Pulumi components built in existing Pulumi languages. If the complexity of your Pulumi YAML projects increases, the new pulumi convert feature can be used to convert your Pulumi YAML program into your favorite general purpose Pulumi language.

Watch our recent workshop: Introduction to Infrastructure as Code with YAML. In this workshop you’ll learn how to build, configure and deploy a modern application using Docker.

Pulumi YAML

Pulumi YAML programs are defined in the same Pulumi.yaml file that all Pulumi projects contain. Instead of specifying a runtime like nodejs or python, Pulumi YAML programs use the yaml runtime. This allows them to also include the set of resources and outputs that their infrastructure requires. All that is needed is the single Pulumi.yaml file.

To get started with Pulumi YAML, run the pulumi new <cloud>-yaml command – templates are available for all of the most popular cloud providers.

The following example shows a Pulumi YAML program that deploys two AWS resources, an AWS S3 Bucket and an S3 Bucket Object to create in that bucket. The program outputs the name of the bucket for use in external tooling.

name: yamldemo
runtime: yaml
resources:
 bucket:
 type: aws:s3:Bucket
 properties:
 website:
 indexDocument: index.html
 index.html:
 type: aws:s3:BucketObject
 properties:
 bucket: ${bucket.id}
 content: <h1>Hello, world!</h1>
 contentType: text/html
 acl: public-read
outputs:
 url: http://${bucket.websiteEndpoint}

This can be deployed with the same familiar pulumi CLI experience - including previews, stacks, configuration and more:

> pulumi up
Updating (dev)
View Live: https://app.pulumi.com/lukehoban/yamldemo/dev/updates/32
Type Name Status
+ pulumi:pulumi:Stack yamldemo-dev created
+ ├─ aws:s3:Bucket bucket created
+ └─ aws:s3:BucketObject index.html created
Outputs:
url: "http://bucket-801e3f9.s3-website-ap-southeast-2.amazonaws.com"
Resources:
+ 3 created
Duration: 8s

And outputs can be used just as in any other Pulumi project:

> curl $(pulumi stack output url)
<h1>Hello, world!</h1>

Why YAML?

Pulumi already supports rich general purpose programming languages, so our goal with Pulumi YAML was to support the other extreme – to be the simplest, most broadly understandable and recognizable, and easiest to generate option for specifying cloud infrastructure. We didn’t want to invent a new proprietary DSL, instead we wanted to support a widely recognized existing markup language.

YAML is the lingua franca of so many existing cloud technologies in use today – from Kubernetes and CloudFormation, to Docker Compose and Serverless Framework. Pulumi YAML takes inspiration from all of these, with a simple document model and direct expression of the Pulumi resource model.

The one key feature of Pulumi YAML is that all property values are implicitly interpolations. This means that any property value can include ${...} to use the output of one resource as part of the input to another resource.

Using YAML does lead to some constraints. There are no loops, no conditionals, no unit testing, and (currently) no rich IDE tooling. All of these are things you can get from any other Pulumi language. But Pulumi YAML explicitly targets simple use cases where these are not needed and avoids the additional complexity of supporting these features.

Use Components Built in Other Pulumi Languages

Pulumi YAML programs can use any package and any resource defined in the Pulumi Registry. This includes 80+ cloud and SaaS providers, comprising tens of thousands of cloud infrastructure resources. Programs can combine resources from multiple providers – such as using AWS, Docker, PostgreSQL and Cloudflare resources from a single infrastructure definition – to describe the complete provisioning of a cloud application’s infrastructure.

Pulumi YAML programs can also use any Pulumi component package defined in another Pulumi language. Pulumi component packages are reusable infrastructure components built in one Pulumi language, and exposed via Pulumi Schema to all other Pulumi languages - including Pulumi YAML.

For example, a complete EKS cluster can be configured using the EKS package (implemented in TypeScript):

name: eks-cluster
runtime: yaml
resources:
 cluster:
 type: eks:Cluster
outputs:
 kubeconfig: ${cluster.kubeconfig}

Organizations can also build their own reusable components in existing Pulumi languages like TypeScript, Python and Go to capture key aspects of their internal design patterns and compliance criteria, and make those available to other developers within the organization to consume from Pulumi YAML. For example, a hypothetical ACMECorp organization could create a high-level serverless application framework built in Go or TypeScript, which would support deploying serverless apps with just a few lines of YAML (note that this results in something similar to Serverless Framework, but as just a simple library defined in Pulumi):

name: serverless-simple-yaml
resources:
 receiverQueue:
 type: aws:sqs:Queue
 svc:
 type: acmecorp:serverless:Service
 properties:
 provider:
 name: aws
 runtime: nodejs10.x
 iam:
 role:
 managedPolicies:
 - arn:aws:iam::aws:policy/AmazonSQSFullAccess
 functions:
 sender:
 handler: handler.sender
 environment:
 QUEUE_NAME: ${receiverQueue.name}
 events:
 - http:
 method: post
 path: sender
 receiver:
 handler: handler.receiver
 events:
 - sqs:
 arn: ${receiverQueue.arn}

Kubernetes Resources

Kubernetes resources are a very common place to use YAML in practice today, and because Pulumi’s Kubernetes provider is a native provider that projects the exact same API as the Kubernetes API Server, this means you can in most cases just copy/paste Kubernetes YAML into a Pulumi YAML program to deploy the Kubernetes resources using the Pulumi deployment engine. For example, everything nested inside properties below is directly copy-pasted from a Kubernetes YAML specification.

name: yamldemo
runtime: yaml
resources:
 pod:
 type: kubernetes:core/v1:Pod
 properties:
 apiVersion: v1
 kind: Pod
 metadata:
 name: debug-network-pod
 spec:
 containers:
 - command:
 - sleep
 - "3600"
 image: praqma/network-multitool
 name: debug-network-container

This allows you to then opt-in to additional features of Pulumi - like passing outputs of one resource into another, waiting for resource readiness before completing an update, passing secret configuration into a deployment, or mixing Kubernetes resources with other cloud infrastructure specifications, such as cloud-native data stores or IAM permissions.

Full Features of the Pulumi Platform

Although most use cases for Pulumi YAML are for simple programs, it’s possible to describe larger collections of infrastructure in Pulumi YAML, using all of the features of the Pulumi Platform.

The following example highlights the use of a variety of Pulumi features via Pulumi YAML to deploy a complete Azure Container Apps application with Pulumi:

Config: Specify the Pulumi configuration parameters and their defaults
Variables: Use Pulumi function invokes to get data from the cloud provider for use in constructing resources.
Providers: Create resource providers to connect to specific instances of a target cloud, like an Azure account or in this case a Docker registry.
Resource Options: Pass any Pulumi Resource Option to a resource definition, to define replacement behavior, resource protection, secret outputs and more.
Multiple Providers: Pass outputs from Azure resources into Docker resources, and outputs from Docker resources back into Azure resources to orchestrate a complete application deployment.

name: azure-container-apps
runtime: yaml
description: Azure Container Apps example
configuration:
 sqlAdmin:
 type: String
 default: pulumi
variables:
 sharedKey:
 Fn::Invoke:
 Function: azure-native:operationalinsights:getSharedKeys
 Arguments:
 resourceGroupName: ${resourceGroup.name}
 workspaceName: ${workspace.name}
 Return: primarySharedKey
 adminUsername:
 Fn::Invoke:
 Function: azure-native:containerregistry:listRegistryCredentials
 Arguments:
 resourceGroupName: ${resourceGroup.name}
 registryName: ${registry.name}
 Return: username
 adminPasswords:
 Fn::Invoke:
 Function: azure-native:containerregistry:listRegistryCredentials
 Arguments:
 resourceGroupName: ${resourceGroup.name}
 registryName: ${registry.name}
 Return: passwords
resources:
 resourceGroup:
 type: azure-native:resources:ResourceGroup
 workspace:
 type: azure-native:operationalinsights:Workspace
 properties:
 resourceGroupName: ${resourceGroup.name}
 sku:
 name: "PerGB2018"
 retentionInDays: 30
 kubeEnv:
 type: azure-native:web:KubeEnvironment
 properties:
 resourceGroupName: ${resourceGroup.name}
 environmentType: Managed
 appLogsConfiguration:
 destination: "log-analytics"
 logAnalyticsConfiguration:
 customerId: ${workspace.customerId}
 sharedKey: ${sharedKey}
 registry:
 type: azure-native:containerregistry:Registry
 properties:
 resourceGroupName: ${resourceGroup.name}
 sku:
 name: "Basic"
 adminUserEnabled: true
 provider:
 type: pulumi:providers:docker
 properties:
 registryAuth:
 - address: ${registry.loginServer}
 username: ${adminUsername}
 password: ${adminPasswords[0].value}
 myImage:
 type: docker:RegistryImage
 properties:
 name: ${registry.loginServer}/node-app:v1.0.0
 build:
 context: ${pulumi.cwd}/node-app
 options:
 provider: ${provider}
 containerapp:
 type: azure-native:web:ContainerApp
 properties:
 resourceGroupName: ${resourceGroup.name}
 kubeEnvironmentId: ${kubeEnv}
 configuration:
 ingress:
 external: true
 targetPort: 80
 registries:
 - server: ${registry.loginServer}
 username: ${adminUsername}
 passwordSecretRef: "pwd"
 secrets:
 - name: "pwd"
 value: ${adminPasswords[0].value}
 template:
 containers:
 - name: "myapp"
 image: ${myImage.name}
outputs:
 endpoint: https://${containerapp.configuration.ingress.fqdn}

Full Support for the Pulumi Service

For users of the Pulumi Service, including the Individual, Team, Enterprise and Business Critical offerings, all of the features you rely on from the Pulumi Service are available for your Pulumi YAML programs as well.

Your Pulumi YAML projects are shown along with the rest of your Pulumi projects authored in other languages, and each Pulumi YAML project can have multiple stacks.

Policy as Code written in any Pulumi language can be applied to your Pulumi YAML programs, and resources provisioned with Pulumi YAML are deep linked to the corresponding resources in your cloud provider. And most fundamentally, your infrastructure state is managed completely transparently.

YAML as a Compilation Target and Cue Support

Pulumi’s YAML runtime supports both YAML and JSON natively. Both of these formats are easy targets for machine generation of infrastructure specifications, which opens up a wide range of additional interesting use cases for Pulumi YAML. For example, early users have built web-based provisioning portals that generate Pulumi YAML programs on the fly to deploy with Pulumi Automation API.

This also means that the rich set of languages that are designed to compile down to YAML/JSON can be used directly with Pulumi YAML. Teams that prefer to use Cue, Jsonnet, HCL or any other DSL that compiles to YAML/JSON can be used with Pulumi YAML.

For example, this Cue program defines a collection of Azure cloud infrastructure resources using Pulumi YAML.

In Pulumi.yaml:

name: cuedemo
runtime:
 name: yaml
 options:
 compiler: cue export

In main.cue:

package main

resources: {
 (#StaticWebApp & {
 _name: "bob"
 }).resources

 (#StaticWebApp & {
 _name: "anna"
 _documents: {
 root: "website"
 index: "index.html"
 }
 }).resources
}

#StaticWebApp: {
 _name: string
 _documents: {
 root: string | *"website"
 index: string | *"index.html"
 error404: string | *"404.html"
 }

 resources: {
 "\(_name)-group":
 type: "azure-native:resources:ResourceGroup"

 "\(_name)-storage": {
 type: "azure-native:storage:StorageAccount"
 properties: {
 resourceGroupName: "${\(_name)-group.name}"
 kind: "StorageV2"
 sku: {
 name: "Standard_LRS"
 }
 }
 }

 "\(_name)-website": {
 type: "azure-native:storage:StorageAccountStaticWebsite"
 properties: {
 resourceGroupName: "${\(_name)-group.name}"
 accountName: "${\(_name)-storage.name}"
 indexDocument: "\(_documents.index)"
 error404Document: "\(_documents.error404)"
 }
 }

 "\(_name)-\(_documents.index)": {
 type: "azure-native:storage:Blob"
 properties: {
 resourceGroupName: "${\(_name)-group.name}"
 accountName: "${\(_name)-storage.name}"
 containerName: "${\(_name)-website.containerName}"
 contentType: "text/html"
 type: "Block"
 source: {
 "Fn::FileAsset": "./\(_documents.root)/\(_documents.index)"
 }
 }
 }
 }
}

Running pulumi up will now automatically compile the Cue into JSON and deploy it - no need to manually run a transpiler.

> pulumi up
Previewing update (dev)
View Live: https://app.pulumi.com/lukehoban/cuedemo/dev/previews/7ac07d0c-262e-41c8-a2c7-6d3f2734cf95
Type Name Plan
+ pulumi:pulumi:Stack cuedemo-dev create
+ ├─ azure-native:resources:ResourceGroup bob-group create
+ ├─ azure-native:resources:ResourceGroup anna-group create
+ ├─ azure-native:storage:StorageAccount anna-storage create
+ ├─ azure-native:storage:StorageAccount bob-storage create
+ ├─ azure-native:storage:StorageAccountStaticWebsite anna-website create
+ ├─ azure-native:storage:StorageAccountStaticWebsite bob-website create
+ ├─ azure-native:storage:Blob anna-index.html create
+ └─ azure-native:storage:Blob bob-index.html create
Resources:
+ 9 to create

Deploy infrastructure with Pulumi YAML

Define your cloud resources in a single Pulumi.yaml file, combine multiple providers, and reuse components built in other Pulumi languages.

Get started

Convert to Other Pulumi Languages

When the complexity of infrastructure managed by a Pulumi YAML program grows beyond what can be managed easily with YAML, users have a variety of options to “grow up” without disruption to their infrastructure.

First and foremost, complexity can be pushed into a reusable component defined in another Pulumi language, and then used from Pulumi YAML, to keep the interface simple and clean for Pulumi YAML consumers.

But for cases where the YAML program itself does need to get more complex, using languages like Cue or Jsonnet provides an easy path, similar to what is frequently found in the Kubernetes ecosystem as complexity increases. This often introduces expressivity limitations and slightly cumbersome templating as you continue to scale through.

So to give the best of both worlds, Pulumi YAML also supports the option to convert your Pulumi YAML programs into programs defined in any other Pulumi language. For example, the original example above can be converted to a Python program with just a single command:

> pulumi convert --language python --out ./bucket-py

This will generate a new Python project in the ./bucket-py/ folder which specifies the exact same infrastructure so that you can cd into that folder, pulumi up and see no changes. The __main__.py generated by pulumi convert looks exactly like what you would have written in Python.

import pulumi
import pulumi_aws as aws

bucket = aws.s3.Bucket("bucket", website=aws.s3.BucketWebsiteArgs(
 index_document="index.html",
))
indexhtml = aws.s3.BucketObject("index.html",
 bucket=bucket.id,
 content="<h1>Hello, world!</h1>",
 content_type="text/html",
 acl="public-read")
pulumi.export("url", bucket.website_endpoint)

From here, you can adopt more features available in Pulumi Python - loops and conditionals, testing, components, and much more!

Summary

We are really excited to be introducing YAML to the collection of languages supported by Pulumi’s Universal Infrastructure as Code platform. With the addition of Pulumi YAML, every cloud builder now has a great option for deploying cloud infrastructure using Pulumi. All of the benefits of Pulumi are available from YAML - the richest Native Providers for major cloud platforms, a wide range of higher-level components in the Pulumi Registry, deep support for secrets, infrastructure refactoring with aliases, a long tail of Cloud and SaaS providers, enforcement of Policy as Code and Audit Logs for all deployments in the Pulumi Service, and so much more. Because you can use components built with other Pulumi languages, the interface to Pulumi YAML can be kept simple, allowing platform teams to define simple interfaces to their cloud infrastructure platform. But if and when the Pulumi YAML does get more complex than is appropriate for YAML, the pulumi convert command provides an easy path to continue to get the full benefits of Pulumi’s general purpose programming language options.

The Pulumi YAML language support is open source at https://github.com/pulumi/pulumi-yaml, and YAML examples are available for all packages and cloud providers in the Pulumi Registry. You can get started with Pulumi YAML today in the Pulumi Getting Started guide and the Pulumi YAML language docs. We can’t wait to see what you build!

Generate Kubernetes YAML with Familiar Programming Languages

Levi Blackstone — Tue, 03 Mar 2020 00:00:00 +0000

Stop writing Kubernetes YAML by hand, and start using the power of familiar programming languages! Pulumi can generate Kubernetes manifests that easily integrate into existing CI/CD workflows.

While Pulumi has excellent support for deploying and updating Kubernetes resources on a cluster, many users have asked for the option to render YAML that they can integrate into existing workflows. The v1.5.4 release of pulumi-kubernetes adds the renderYamlToDirectory option, which enables this feature. This option is available in every Pulumi-supported language, including TypeScript/JavaScript, Python, and .NET (Go support is coming soon!).

Aside from easily templating configuration across resources, using a familiar programming language allows you to write and consume libraries, and easily mix in infrastructure configuration (e.g., managed database endpoints, object storage, etc.), all in the same program.

Rendering YAML from TypeScript

First, choose a directory for the rendered manifests, and specify that path on a Provider.

import * as k8s from "@pulumi/kubernetes";
import * as kx from "@pulumi/kubernetesx";

// Instantiate a Kubernetes Provider and specify the render directory.
const provider = new k8s.Provider("render-yaml", {
 renderYamlToDirectory: "rendered",
});

Next, use that Provider for any Kubernetes resources you want to render as YAML.

// Create a Kubernetes PersistentVolumeClaim.
const pvc = new kx.PersistentVolumeClaim("data", {
 spec: {
 accessModes: [ "ReadWriteOnce" ],
 resources: { requests: { storage: "1Gi" } },
 }
}, { provider });

// Create a Kubernetes ConfigMap.
const cm = new kx.ConfigMap("cm", {
 data: { "config": "very important data" },
}, { provider });

// Create a Kubernetes Secret.
const secret = new kx.Secret("secret", {
 stringData: {
 "password": new random.RandomPassword("pw", {
 length: 12}).result,
 }
}, { provider });

// Define a Pod.
const pb = new kx.PodBuilder({
 containers: [{
 env: {
 CONFIG: cm.asEnvValue("config"),
 PASSWORD: secret.asEnvValue("password"),
 },
 image: "nginx",
 ports: {http: 8080},
 volumeMounts: [ pvc.mount("/data") ],
 }]
});

// Create a Kubernetes Deployment.
const deployment = new kx.Deployment("nginx", {
 spec: pb.asDeploymentSpec( { replicas: 3 } ),
}, { provider });

// Create a Kubernetes Service.
const service = deployment.createService({
 type: kx.types.ServiceType.LoadBalancer,
});

Now, run pulumi update, and Pulumi renders these resources to YAML. The update process resolves Outputs as usual, so that the manifests can include other infrastructure configuration specified in your program. The rendered manifests are kept in sync with changes to the program on each update.

Here’s what the resulting directory looks like:

Note that CustomResourceDefinition resources need to be applied first, so they are rendered in a separate subdirectory. (This example doesn’t include any CRDs, so the directory is empty). You could deploy the rendered manifests with kubectl like this:

kubectl apply -f "${RENDER_DIRECTORY}/0-crd"
kubectl apply -f "${RENDER_DIRECTORY}/1-manifest"

Voilà! From 33 (44 with whitespace and comments) lines of TypeScript code to 102 lines of YAML!

Here’s the rendered Deployment resource.

See this gist for the complete rendered output.

Caveats

There are two important caveats to note about YAML rendering support:

The YAML-rendered resources are not created on a Kubernetes cluster, so information that is computed server-side will not be available in your program. For example, a Service will not have IP assignments, so attempting to export these values will not work as usual (i.e., the value will be undefined).
Any Secret values will appear in plaintext in the rendered manifests. This includes any values marked as secret in Pulumi. A warning will be printed for any secret values being rendered to YAML, but it is your responsibility to protect the rendered files.

Learn More

If you’d like to learn about Pulumi and how to manage your infrastructure and Kubernetes through code, get started today. Pulumi is open source and free to use.

For further examples on how to use Pulumi to create Kubernetes clusters, or deploy workloads to a cluster, check out the rest of the Kubernetes tutorials.

As always, you can check out our code on GitHub, follow us on Twitter, subscribe to our YouTube channel, or join our Community Slack channel if you have any questions, need support, or just want to say hello.

Pulumi Blog: Yaml

New: Helm Chart v4 resource with new features and languages

What’s new

New language support - Java SDK & YAML SDK

OCI registry support

Lock file support

Better handling of chart values

Better connectivity

Improved resource ordering

New-style Pulumi transformations

Post-rendering support

“Keep” policy

Release name

Limitations

Not supported: Kubernetes transformations

Not supported: Helm hooks

Evolving: Resource outputs

Example: Argo CD

Conclusion

New: ConfigGroup, ConfigFile resources for Java, YAML SDKs

kubernetes.yaml/v2.ConfigGroup

Manifest Files

YAML Text

Object Literals

Resource Ordering

Resource Prefixes

Transformation Support

kubernetes.yaml/v2.ConfigFile

Manifest File

Example: Certificate Manager

Conclusion

How to Create and Share a Pulumi Template

Step 1: Create a new project

Step 2: Design and build the program

Step 3: Turn the project into a template

Step 4: Test-drive the template locally

Step 5: Publish the template

Bonus step: Add a Deploy with Pulumi button

See it in action

Wrapping up

Announcing: Pulumi Deployments, YAML GA, Arch Templates

Pulumi Deployments

Pulumi YAML

Architecture Templates

Project Level Configuration

Conclusion

Pulumi YAML General Availability

Why Pulumi YAML?

Pulumi YAML 1.0

Full support for pulumi convert

IDE Support (Preview)

Type Checking and Diagnostics

Simplified function syntax

New str Package

Pulumi Kubernetes Operator Inline Program Support

Conclusion

Create an AWS Static Website Fast with Angular and Pulumi

What is Angular?

What are Components in Angular?

What is the AWS Static Website Component?

The Angular Components

Building the Pulumi Café Website

Deploy the Site with Pulumi

Modify the .yaml File

Manage Your Site with Pulumi

Learn More

Deploy Static Websites to AWS using 10 lines of YAML

What the Component Does

Some Examples

Create a Website

Get Started

Deploy the Website

What You Gain

Input Properties

Output Properties

Add a CloudFront CDN

More Pulumi Advantages

Learn More

Install Pulumi

Shared configuration stacks with AWS Systems Manager

Full support for `pulumi convert`

New `str` Package

Pulumi Kubernetes Operator Inline `Program` Support