Pulumi Blog: Alex Mullans

Dec. 8 releases: K8s examples, exclude protected resources from destroy, easier invites to the Pulumi Service

Alex Mullans — Wed, 08 Dec 2021 08:00:00 -0700

With the holiday season approaching, we’ve been focused on tidying up our products, delivering asks we’ve heard from you in GitHub and at conferences, and looking ahead to 2022! Read on to learn about what’s new this release:

Pulumi Registry, Pulumi Packages, & integrations
Pulumi CLI and core technologies
- Exclude protected resources from pulumi destroy
- Pulumi Package plugins can now be hosted on GitHub Releases
Pulumi Service & Pulumi.com
- More ways to share invites to your organization
- Self-Hosted Pulumi Service can disable email sign-up and login

Pulumi Registry, Pulumi Packages, & integrations

Helm `Release` resource supports local charts

When you’re developing or using a Helm chart stored on your local machine, it can be helpful to use a Pulumi program to deploy and test it. Now, you can! The Release resource’s constructor now accepts paths to local Helm charts.

Learn more in this GitHub issue

New examples for the Kubernetes native provider

We’ve added additional examples to our native provider for Kubernetes to make it easier to get started with common resources. Jump into Pulumi Registry with the links below:

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const my_service = new kubernetes.core.v1.Service("my_service", {
 spec: {
 selector: {
 app: "MyApp",
 },
 ports: [{
 protocol: "TCP",
 port: 80,
 targetPort: 9376,
 }],
 },
});

import pulumi
import pulumi_kubernetes as kubernetes

my_service = kubernetes.core.v1.Service(
 "my_service",
 spec=kubernetes.core.v1.ServiceSpecArgs(
 selector={
 "app": "MyApp",
 },
 ports=[kubernetes.core.v1.ServicePortArgs(
 protocol="TCP",
 port=80,
 target_port=9376,
 )],
 ))

using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

class MyStack : Stack
{
 public MyStack()
 {
 var service = new Kubernetes.Core.V1.Service("my_service", new Kubernetes.Types.Inputs.Core.V1.ServiceArgs
 {
 Spec = new Kubernetes.Types.Inputs.Core.V1.ServiceSpecArgs
 {
 Selector =
 {
 { "app", "MyApp" },
 },
 Ports =
 {
 new Kubernetes.Types.Inputs.Core.V1.ServicePortArgs
 {
 Protocol = "TCP",
 Port = 80,
 TargetPort = 9376,
 },
 },
 },
 });
 }
}

package main

import (
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := corev1.NewService(ctx, "my_service", &corev1.ServiceArgs{
 Spec: &corev1.ServiceSpecArgs{
 Selector: pulumi.StringMap{
 "app": pulumi.String("MyApp"),
 },
 Ports: corev1.ServicePortArray{
 &corev1.ServicePortArgs{
 Protocol: pulumi.String("TCP"),
 Port: pulumi.Int(80),
 TargetPort: pulumi.Int(9376),
 },
 },
 },
 })
 if err != nil {
 return err
 }
 return nil
 })
}

New resources in the AWS Native provider

We shipped new versions of the AWS Native provider (0.5.0 through 0.8.0) that added 30 new resources giving you access to new resources in feature areas like EC2, Evidently, Lex, and more.

New resources in the Azure Native provider

We shipped new versions of the Azure Native provider (1.46.0 through 1.49.0) that collectively added 8 new resources.

Azure Native version 1.47.0 fixes a security issue that could result in your Azure credentials being stored in your state in cleartext. Read the description of the issue to see if you’re affected and how to remediate any cleartext secrets.

As a precaution, we recommend updating to the latest version of the Azure Native provider.

Support for Kubernetes 1.23

The Pulumi Kubernetes native provider now supports the latest version of Kubernetes that was released on December 7th: version 1.23. See the Kubernetes v1.23.0 changelog to learn about the new Kubernetes features and updates you can use via our Kubernetes native provider.

Pulumi CLI and core technologies

In this milestone, we shipped Pulumi versions 3.17.1 through 3.19.0. The full list of changes in each version is available in the changelog; read on to learn about some of the biggest changes.

Exclude protected resources from `pulumi destroy`

In some scenarios, you may want to retain certain resources even while destroying the rest of the resources in a stack. Previously, you had to do this by providing a list of every resource URN that you did want to destroy to the pulumi destroy command. Now, you can use the --exclude-protected argument to destroy all unprotected resources.

Learn more in this GitHub issue

Pulumi Package plugins can now be hosted on GitHub Releases

To publish a Pulumi Package, you need to publish the resource provided plugin that contains the implementation of your package. Previously, you couldn’t use popular release services like GitHub Releases because the pluginDownloadUrl format in the Pulumi Packages schema was too strict.

Now, you can construct a pluginDownloadUrl with placeholders for ${VERSION}, ${OS}, and/or ${ARCH} (case-sensitive) and these values will be interpolated whenever the Pulumi CLI attempts to download a Pulumi Package’s plugin.

Learn more in this GitHub PR

Pulumi Service & Pulumi.com

Previously, inviting new members to your Pulumi Service organization meant entering each of their email addresses and letting Pulumi send them an invitation email. Now, you can click or tap the “Copy new invite link” button on the Members page and get a one-time use link that you can send to your invitee however you like.

Once you’ve connected your Self-Hosted Pulumi Service instance with a directory–a SAML directory like Azure Active Directory or a source control directory like GitHub, GitLab, or Bitbucket–you may want to remove the ability for your users to sign in without using that directory. Now, you can by setting the PULUMI_DISABLE_EMAIL_LOGIN and PULUMI_DISABLE_EMAIL_SIGNUP environment variables to true.

See the self-hosted setup guide for more information.

Nov. 17 releases: `dependOn` Helm charts, new Elastic Cloud provider, functions support outputs, set the CLI's default organization

Alex Mullans — Wed, 17 Nov 2021 08:00:00 -0700

It’s been another exciting few weeks here at Pulumi! We’ve caught our breath from Cloud Engineering Summit (don’t forget to check out the talks if you haven’t yet!) and we’re back to adding new value and highly-requested fixes across the Pulumi Cloud Engineering Platform. Read on to learn about new providers, new enhancements to the core Pulumi experience, and more!

Pulumi Registry, Pulumi Packages, & integrations
Pulumi CLI and core technologies
Pulumi Service & Pulumi.com
- Set the default organization for newly-created stacks

Pulumi Registry, Pulumi Packages, & integrations

`dependsOn` can now await a fully-deployed Helm `Chart` resource

Pulumi’s dependsOn option enables you to write programs that deploy resources in a specific order. Previously, using this option with a Helm Chart resource could be confusing because Pulumi would consider the resource ready once it had registered the chart, even if Helm was still deploying some of the resources defined by the chart. Now, you can depend on a new ready attribute that will ensure that none of the dependent resources are deployed until the chart is deployed. For example, the code sample below creates an NGINX Ingress Controller and then creates a Kubernetes ConfigMap that depends on the controller being fully deployed.

import * as k8s from "@pulumi/kubernetes";

const nginxIngress = new k8s.helm.v3.Chart("nginx-ingress", {
 chart: "nginx-ingress",
 version: "1.24.4",
 namespace: "test-namespace",
 fetchOpts:{
 repo: "https://charts.helm.sh/stable",
 },
});

// Create a ConfigMap depending on the Chart. The ConfigMap will not be created until after all of the Chart
// resources are ready. Note the use of the `ready` attribute; depending on the Chart resource directly will not work.
new k8s.core.v1.ConfigMap("foo", {
 metadata: { namespace: namespaceName },
 data: {foo: "bar"}
}, {dependsOn: nginxIngress.ready})

This change impacts the older Helm Chart resource. The new Helm Release resource always waits for the Helm chart to be fully deployed.

Learn more about dependsOn and Helm charts in this GitHub issue

New provider: Elastic Cloud

You can now provision Elastic Cloud resources like ElasticSearch using Pulumi! Visit the ElasticCloud (EC) provider on Pulumi Registry to get started. The example below shows how you can create an Elastic Cloud deployment in just a few lines of code:

const ec = require("@pulumi/ec")

const REGION = "us-east-1";

const latestVersion = elasticCloud.getStack({
 region: REGION,
 versionRegex: "latest"
});

new elasticCloud.Deployment('my-deployment', {
 region: REGION,
 version: latestVersion.then((x: { version: string; }) => x.version),
 deploymentTemplateId: "aws-io-optimized-v2",
 elasticsearch: {}
});

import * as elasticCloud from "@pulumi/ec";

const REGION = "us-east-1";

const latestVersion = elasticCloud.getStack({
 region: REGION,
 versionRegex: "latest"
});

new elasticCloud.Deployment('my-deployment', {
 region: REGION,
 version: latestVersion.then((x: { version: string; }) => x.version),
 deploymentTemplateId: "aws-io-optimized-v2",
 elasticsearch: {}
});

import pulumi_ec as ec

REGION = "us-east-1"

latest_version = ec.get_stack(
 region=REGION, version_regex="latest").version

deployment = ec.Deployment('my-deployment', region=REGION, version=latest_version,
 deployment_template_id="aws-io-optimized-v2", elasticsearch={})

import (
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
 ec "github.com/pulumi/pulumi-ec/sdk/go/ec"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 version, err := ec.GetStack(ctx, &GetStackArgs{
 Region: pulumi.String("us-east-1"),
 VersionRegex: "latest",
 }
 if err != nil {
 return err
 }

 _, err := ec.NewDeployment(ctx, "my-deployment", &ec.DeploymentArgs{
 Region: pulumi.String("us-east-1"),
 Version: version.Version,
 DeploymentTemplateId: pulumi.String("aws-io-optimized-v2")
 ElasticSearch: &ec.DeploymentElasticsearch{},
 })
 if err != nil {
 return err
 }

 return nil
 })
}

using System.Collections.Generic;
using System.Threading.Tasks;
using Pulumi;
using ElasticCloud = Pulumi.ElasticCloud;

class Program
{
 static Task Main() =>
 Deployment.Run(() => {
 var latest = Output.Create(ElasticCloud.GetStack.InvokeAsync(new ElasticCloud.GetStackArgs
 {
 Region = "us-east-1",
 VersionRegex = "latest",
 });

 var deployment = ElasticCloud.Deployment("my-deployment", new ElasticCloud.DeploymentArgs
 {
 Region = "us-east-1",
 Version = latest.Version,
 DeploymentTemplateId = "aws-io-optimized-v2",
 ElasticSearch = {},
 });
 });
}

AWS Native provider supports auto-naming

Most Pulumi providers support auto-naming for resources, which makes it easier to have multiple stacks of the same Pulumi program by avoiding naming conflicts, among other benefits. Now, our AWS Native provider joins the club with full auto-naming support.

Learn more about AWS auto-naming in this GitHub issue

New resources in the AWS Native provider

We shipped new versions of the AWS Native provider (0.3.0 through 0.4.0) that added 38 new resources giving you access to resources like AWS::EKS::Cluster, AWS::Lightsail::Database, and several new resources in the AWS::Redshift namespace.

New resources in the Azure Native provider

We shipped new versions of the Azure Native provider (1.43.0 through 1.45.0) that collectively added 4 new resources giving you access to resources like Cognitive Services.

Pulumi CLI and core technologies

In this milestone, we shipped Pulumi version 3.17.0. The full list of changes in this version is available in the changelog; read on to learn about some of the biggest changes.

Functions now support outputs

When creating complex infrastructure, it’s common to pass information about one resource to a function provided by another resource. Previously, doing this in Pulumi required you to use apply to get the information from the first resource and pass it into the function. Now, we’ve simplified the process so you can reference outputs directly from the function—no apply needed. To take advantage of these changes, make sure you’ve updated to the latest versions of Pulumi and the providers you’re using.

Learn more in the “Functions now accept outputs” blog post

`--json` flag for `pulumi {update|destroy|preview|refresh}`

When automating or troubleshooting Pulumi operations, it can be helpful to see all of the details emitted by the Pulumi engine rather than the standard progress bar. Now, you can! Just add --json to your pulumi {update|destroy|preview|refresh} command.

Learn more about the new --json flag in this GitHub issue

Python 3.6 support will be deprecated soon

Python 3.6 will be end-of-lifed (“EOLed”) on December 23, 2021. Around that time, Pulumi will drop support for Python 3.6. If you’re using Python 3.6, consider upgrading to a newer version of Python. The Pulumi CLI will display a message when it runs a Pulumi program using Python 3.6.

Learn more about Python 3.6 deprecation in this GitHub issue

Pulumi Service & Pulumi.com

Set the default organization for newly-created stacks

If you’re using Pulumi as part of a team or company, rather than as part of a personal project, you likely want to ensure that any infrastructure you deploy is registered in your company’s Pulumi organization, rather than in your personal Pulumi account. Previously, you needed to either create each stack inside the organization (pulumi new -s <org>/<stackname>) or transfer stacks into the organization once created. Now, you can set the default organization for Pulumi to use when creating new stacks: pulumi org set-default <org>. Watch the asciinema recording below to get a feel for how it works:

Learn more about setting the default organization in this GitHub issue

Oct. 27 releases: Pulumi Registry, K8s Operator 1.0

Alex Mullans — Wed, 27 Oct 2021 08:00:00 -0700

It’s been an incredibly busy few weeks at Pulumi. We’ve met thousands of you at our second annual Cloud Engineering Summit, shipped major new releases like Pulumi Registry and Pulumi Kubernetes Operator 1.0 and continued to innovate on and improve the entire Pulumi Cloud Engineering Platform. Get caught up on Cloud Engineering Summit to learn what’s coming in cloud engineering, or read on for this month’s news.

Cloud Engineering Summit
Pulumi Registry, Pulumi Packages, & integrations
Pulumi CLI and core technologies
- Resource methods for Pulumi Packages
- Automation API: delete, unprotect state in .NET
Pulumi Service & Pulumi.com
- Automated audit log export

Cloud Engineering Summit

On October 20th, thousands of you joined us for our second annual Cloud Engineering Summit. We heard from industry leaders like Pulumi CEO Joe Duffy, Google’s Kelsey Hightower, AWS’ Emily Freeman, Microsoft’s Adora Nwodo, and many more on topics from modern reference architectures for Kubernetes to migrating a monolith to cloud-native. You can watch these and all Cloud Engineering Summit sessions on our website–make sure to catch any sessions you missed!

Watch Cloud Engineering Summit sessions on-demand

Pulumi Registry, Pulumi Packages, & integrations

Pulumi Registry: your window to the cloud

As the Pulumi community has grown, we’ve heard from many of you that you needed a central hub where you could find all the cloud resources you can build with Pulumi. Pulumi Registry is that place: a searchable collection of Pulumi Packages published by Pulumi and our partners. With Pulumi Registry, you can easily find the package with the resources you need, install that package directly into your project, and start building. You can choose from Providers that give you full access to everything a cloud provider has to offer, or choose a Component that gets you started quickly with best practices and sensible defaults baked in.

Visit Pulumi Registry or read the announcement.

New Pulumi Components for AWS, Azure, and popular Helm charts

As part of the launch of Pulumi Registry, we’ve also created new Pulumi Components that can help you get started quickly with common cloud infrastructure, including:

AWS services like API Gateway, VPC, Redshift, and Aurora
Azure Container Registry
Popular Helm charts including cert-manager, CoreDNS, and NGINX

Visit Visit Pulumi Registry and filter to Components to get started.

Pulumi Kubernetes Operator 1.0

The Pulumi Kubernetes Operator defines a Kubernetes Custom Resource called pulumi.com/v1/Stack, which represents a Pulumi stack. Using the Operator, you can specify the desired state of your cloud infrastructure using resources managed directly in your Kubernetes cluster, which trigger creation, update and deletion of the detailed cloud infrastructure you need. Get started with the 1.0 release of the Pulumi Kubernetes Operator today!

This example Kubernetes resource uses the Pulumi Kubernetes Operator to deploy a Pulumi program which provisions cloud infrastructure to manage NGINX provided in the target Git repo and commit, and future updates to the commit or config settings will deploy updates to the infrastructure:

apiVersion: pulumi.com/v1
kind: Stack
metadata:
 name: nginx-k8s-stack-production
spec:
 stack: acmecorp/nginx/production
 projectRepo: https://github.com/acmecorp/pulumi-nginx
 commit: 2b0889718d3e63feeb6079ccd5e4488d8601e353

Read the 1.0 announcement for more info

New resources in the AWS Native provider

We shipped a new version of the AWS Native provider (0.2.0) that added 19 new resources giving you access to resources like AWS::ApiGateway::Authorizer, AWS::IAM::Role, and AWS::Lightsail::Instance.

New resources in the Azure Native provider

We shipped new versions of the Azure Native provider (1.35.0 through 1.42.0) that collectively added 31 new resources giving you access to new Azure offerings like vSphere integration, Chaos engineering, and more.

Pulumi CLI and core technologies

In this milestone, we shipped Pulumi versions 3.14.0 through 3.16.0. The full list of changes in each version is available in the changelog; read on to learn about some of the biggest changes.

Resource methods for Pulumi Packages

When authoring component resources you want to publish in a Pulumi Package, it’s often useful to provide additional functionality through methods on the component. Resource methods are similar to functions, but instead of being exposed as top-level functions in a module, methods are exposed as methods on a resource class. This allows for a more object-oriented approach to exposing functionality: operations performed by a resource (that potentially use the resource’s state) can now be exposed as methods on the resource. Resource methods can be implemented once, in your language of choice, and made available to users in all Pulumi languages.

For example, the Cluster component in the eks package has a getKubeconfig method that can be used to generate a kubeconfig for authentication with the cluster that does not use the default AWS credential provider chain, but is instead scoped based on the passed-in arguments. With the new support for resource methods for Pulumi Packages, we can make this method available to all the Pulumi languages, which is exactly what we’ve done in pulumi-eks v0.34.0.

Learn how to author your own methods in the blog post.

Automation API: delete, unprotect state in .NET

In the Automation API in .NET, you can now use new DeleteAsync, UnprotectAsync, and UnprotectAllAsync methods to delete resources from a stack’s state (like running pulumi state delete) or unprotect resources (like running pulumi state unprotect) so they can be deleted.

await stack.State.DeleteAsync(resource.Urn);

await stack.State.UnprotectAsync(resource.Urn);

Learn more in this GitHub PR

Pulumi Service & Pulumi.com

Automated audit log export

Some larger companies manage their security risk using a Security Information and Event Management (SIEM) system to collect security information from all of the systems and SaaS products they use. To make it easier for you to bring Pulumi audit log events into such systems, you can now enable automated export of your audit logs to an Amazon S3 bucket, which many SIEMs can read from automatically.

Learn more in the docs.

Introducing Pulumi Registry: Deploy Cloud Infra Faster

Alex Mullans — Mon, 18 Oct 2021 00:00:00 -0700

Pulumi offers the most complete infrastructure as code platform for building, deploying, and managing modern cloud infrastructure and applications. When you use Pulumi, a vast library of cloud resources—from compute, storage, and other cloud infrastructure to databases to identity providers to monitoring systems—is at your fingertips. As the Pulumi community has grown, we’ve heard from many of you that we needed a central hub where you could find all of those resources. We’ve also heard from Pulumi partners that they wanted a great place to showcase their integrations with Pulumi so that their customers can more easily learn how to use Pulumi to deploy and manage their products.

Today, we’re excited to launch Pulumi Registry, the one place to discover and share everything you can achieve using Pulumi. Pulumi Registry is a searchable collection of Pulumi Packages published by Pulumi and our partners. With Pulumi Registry, you can easily find the package with the resources you need, install that package directly into your project, and start building. You can choose from Providers that give you full access to everything a cloud provider has to offer, or choose a Component that gets you started quickly with best practices and sensible defaults baked in. All Pulumi Packages are available in all Pulumi languages, so you can build your infrastructure using C#, Go, Python, and TypeScript/JavaScript. You’ll also find all of the documentation you need to succeed: from detailed API reference to how-to guides with source code for specific use cases.

Pulumi Registry is launching with support for 64 of the leading cloud providers and modern cloud SaaS offerings, including Amazon Web Services, Microsoft Azure, Google Cloud, Kubernetes, Auth0, CloudFlare, Confluent Cloud, Datadog, DigitalOcean, Docker, GitHub, Kong, MinIO, MongoDB Atlas, PagerDuty, Snowflake, Spot by NetApp, and SumoLogic. Pulumi Registry also features some of the most popular Helm charts wrapped as Pulumi Components so you can take advantage of the most popular cloud native technologies.

We’re excited to enable any developer to take advantage of the power of these offerings, and we can’t wait to see what you achieve with them and Pulumi.

Find & Deploy Cloud Infrastructure with Pulumi Registry

The modern cloud has put more power and more capability to delight customers in developers’ hands than ever before. In Azure, Amazon Web Services, and Google Cloud alone, there are more than 6,300 distinct services, each of which might be the service that gives your team, or your company, the competitive advantage you need to win in the marketplace. But, with all of this power comes cloud complexity: you need to pick the right service, configure it correctly and securely, and integrate it with all of your existing cloud infrastructure.

Pulumi Registry, an integral part of the Pulumi Cloud Engineering Platform, tames this complexity by bringing together everything you can do with the cloud into a single location. You can see all of our Pulumi Packages, including our Native Providers for the largest and most popular clouds; our broad range of Providers for various cloud, database, networking, monitoring, and other offerings; and all the Components that get you to production faster than ever before. You can easily filter packages by these categories or to find packages for your modern cloud use case, including Serverless, Containers, or Kubernetes.

Once you find a package of interest, everything you need to know to build, deploy, and manage cloud infrastructure with that package is at your fingertips. If you’re using a package for the first time or using a service you haven’t used before, Pulumi Registry has tutorials and scenario-specific how-to guides to help you get started. If you want to jump right in, you can install the package–whether you need the NuGet package, the npm package, the Python package, or the Go module–right from Pulumi Registry. If you’re already familiar with the package and need the API reference, it’s only a click away. We’ve also taken this opportunity to make some improvements to the API documentation (which was previously hosted on pulumi.com/docs), like adding a left navigation and type-ahead filter so it’s easier to quickly find the resource or method you’re looking for.

👉 Visit the Pulumi Registry

Pulumi Providers: The Most Complete Cloud IaC Solution

Pulumi Registry’s packages come in two categories: Providers and Components. Providers are direct representations of all of the resources available from your choice of cloud or SaaS provider: everything from Alicloud to Cloudflare to Okta to Yandex. They’re a great choice when you want to work directly with individual cloud resources and define your infrastructure from scratch. (For higher-level building blocks, skip to the next section.)

For the largest and most popular clouds, we’ve created a special type of Provider that’s unique to Pulumi: Native Providers. Native Providers are the most advanced infrastructure-as-code packages available in the market. Each Native Provider is built directly from the underlying API of the cloud provider, so you get complete access to the full resource model of all of the provider’s services: every resource, every property, every method. We update Native Providers within 24 hours of the underlying API being updated, so you never have to wait to try out the latest improvements and new services.

Just a few weeks ago, we launched the AWS Cloud Control Provider, which is built directly from the new AWS Cloud Control APIs and gives you same-day access to every new AWS service. AWS Cloud Control joins our existing Azure Native Provider, Google Native Provider, and Kubernetes Native Provider.

Use Any Terraform Provider with Pulumi Registry

If Pulumi Registry doesn’t yet have a native provider for your preferred cloud service, that’s OK! With Pulumi, you have full access to the Terraform ecosystem and can use any Terraform provider via our Terraform bridge. A number of bridged Terraform providers are already available in Pulumi Registry, including providers for popular services like Auth0, CloudFlare, Datadog, DigitalOcean, GitHub, and PagerDuty.

Create & Publish Custom Pulumi Providers

We also invite the entire Pulumi community to contribute new Providers to Pulumi Registry. You can enable your customers to manage your cloud service or cloud resources using Pulumi’s modern infrastructure-as-code by authoring a Native Provider, or by bridging an existing Terraform provider into the Pulumi ecosystem, and publishing it to the Pulumi Registry. We’ve also seen passionate customers and community members author Native Providers for cloud services with public, well-known APIs! Pulumi Registry will enable fully self-service publishing in 2022, but until then, we’re happy to work with you to publish your Provider.

👉 Build and publish a Pulumi Provider

Pulumi Components: Prebuilt IaC for Faster Cloud Deployments

When you need full control over the cloud resources you’re building, Providers are a great choice, because they give you access to all of the resources, all of the settings, and everything else you need to build and deploy infrastructure from scratch. But, we frequently hear from developers and infrastructure engineers that they need higher-level building blocks that “just work”: things like secure and production-ready Kubernetes clusters or complete serverless configurations for popular combos like AWS Lambda and API Gateway or Azure Functions and API Management. Pulumi Components are the answer to those requests.

Components are higher-level building blocks that come with best practices and sensible defaults built in. They frequently encode the opinions of the component author so that you don’t have to configure every setting and understand every detail of all the cloud resources you want to use. Let’s look at some examples to understand the power of Components.

Deploy a Production-Ready EKS Cluster with Pulumi (in 15 Lines)

Amazon’s Elastic Kubernetes Service (EKS) is a great example of a powerful but complex service. With Pulumi’s AWS Cloud Control Provider, you can define a production-ready EKS cluster, but you need to know a fair amount about EKS. If you’re a platform team building out a custom Kubernetes shared services platform for a big organization, that’s OK, as you’ll likely want (and expect) full control over every setting on your cluster. For many other scenarios, you just want a well-configured EKS cluster with minimal configuration. Pulumi’s EKS Component gives you just that, with just 15 lines of code:

import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";

// Create a VPC for our cluster.
const vpc = new awsx.ec2.Vpc("my-vpc", {});

// Create an EKS cluster inside of the VPC.
const cluster = new eks.Cluster("my-cluster", {
 vpcId: vpc.id,
 publicSubnetIds: vpc.publicSubnetIds,
 privateSubnetIds: vpc.privateSubnetIds,
 nodeAssociatePublicIpAddress: false,
});

// Export the cluster's kubeconfig.
export const kubeconfig = cluster.kubeconfig;

import pulumi
import pulumi_eks as eks

# Create or lookup a VPC for our cluster.
vpc = ...

# Create an EKS cluster inside of our VPC.
cluster = eks.Cluster('my-cluster',
 eks.ClusterArgs(
 vpc_id = vpc.id,
 public_subnet_ids = vpc.public_subnet_ids,
 private_subnet_ids = vpc.private_subnet_ids,
 node_associate_public_ip_address = False,
 )
)

# Export the cluster's kubeconfig.
pulumi.export('kubeconfig', cluster.kubeconfig)

package main

import (
 "github.com/pulumi/pulumi-eks/sdk/go/eks"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create or lookup a VPC for our cluster.
 vpc := ...

 // Create an EKS cluster with the default configuration.
 cluster, err := eks.NewCluster(ctx, "my-cluster", &eks.ClusterArgs{
 VpcId: vpc.ID,
 PublicSubnetIds: vpc.PublicSubnetIds,
 PrivateSubnetIds: vpc.PrivateSubnetIds,
 NodeAssociatePublicIpAddress: pulumi.Bool(false),
 })
 if err != nil {
 return err
 }

 // Export the cluster's kubeconfig.
 ctx.Export("kubeconfig", cluster.Kubeconfig)
 return nil
 })
}

using Pulumi;
using Pulumi.Eks;

class MyStack : Stack
{
 public MyStack()
 {
 // Create or lookup a VPC for our cluster.
 var vpc = ...;

 // Create an EKS cluster with the default configuration.
 var cluster = new Cluster("my-cluster", new ClusterArgs {
 VpcId = vpc.Id,
 PublicSubnetIds = vpc.PublicSubnetIds,
 PrivateSubnetIds = vpc.PrivateSubnetIds,
 NodeAssociatePublicIpAddress = false,
 });

 // Export the cluster's kubeconfig.
 this.Kubeconfig = cluster.Kubeconfig;
 }

 [Output]
 public Output<object> Kubeconfig { get; set; }
}

Build a Serverless App with AWS Lambda & API Gateway in 21 Lines

For event-driven and stateless workloads, serverless compute can be a great choice that enables you to pay only for the compute time you need instead of keeping VMs or containers idle. But, setting up serverless functions and the networking required for them to work together isn’t always straightforward. Pulumi’s AWS API Gateway package makes it easy to create a serverless function (or several!) and make them accessible via an API Gateway:

import * as apigateway from "@pulumi/aws-api-gateway";
import * as aws from "@pulumi/aws";

const f = new aws.lambda.CallbackFunction("f", {
 callback: async (ev, ctx) => {
 console.log(JSON.stringify(ev));
 return {
 statusCode: 200,
 body: "goodbye",
 };
 },
});

const api = new apigateway.RestAPI("api", {
 routes: [{
 path: "/",
 method: apigateway.Method.GET,
 eventHandler: f,
 }],
});

export const url = api.url;

import pulumi
import pulumi_aws_api_gateway as apigateway

api = apigateway.RestAPI('api', routes=[
 apigateway.RouteArgs(path="/", method=apigateway.Method.GET, event_handler=f),
 apigateway.RouteArgs(path="/www", method=apigateway.Method.GET, local_path="www", index=False),
 apigateway.RouteArgs(path="/integration",
 target=apigateway.TargetArgs(uri="https://www.google.com", type=apigateway.IntegrationType.Http_proxy))
])

pulumi.export('url', api.url)

package main

import (
 "github.com/pulumi/pulumi-aws-api-gateway/sdk/go/apigateway"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create cluster with default settings
 api, err := apigateway.NewRestAPI(ctx, "api", &apigateway.RestAPIArgs{
 Routes: []apigateway.RouteArgs{
 &apigateway.RouteArgs{
 Path: pulumi.String("/"),
 Method: apigateway.MethodGET,
 EventHandler: f,
 },
 &apigateway.RouteArgs{
 Path: pulumi.String("/"),
 Method: apigateway.MethodGET,
 LocalPath: pulumi.String("www"),
 Index: pulumi.Bool(false),
 },
 &apigateway.RouteArgs{
 Path: pulumi.String("/integration"),
 Method: apigateway.MethodGET,
 Target: &apigateway.TargetArgs{
 Uri: pulumi.String("https://www.google.com"),
 Type: apigateway.IntegrationType.Http_proxy,
 },
 },
 }
 })
 if err != nil {
 return err
 }

 ctx.Export("url", api.Url)
 return nil
 })
}

using Pulumi;
using ApiGateway = Pulumi.AwsApiGateway;

class MyStack : Stack
{

 [Output("url")] Output<string> Url { get; set; }

 public MyStack()
 {
 var api = new ApiGateway.RestAPI("api", new ApiGateway.RestAPIArgs
 {
 Routes =
 {
 new ApiGateway.RouteArgs{ Path = "/", Method = ApiGateway.Method.GET, EventHandler = f },
 new ApiGateway.RouteArgs{
 Path = "/www",
 Method = ApiGateway.Method.GET,
 LocalPath = "www",
 Index = false
 },
 new ApiGateway.RouteArgs{
 Path = "/integration",
 Target = new ApiGateway.TargetArgs{
 Uri = "https://www.google.com",
 Type = ApiGateway.IntegrationType.Http_proxy,
 }
 },
 },
 });

 this.Url = api.Url;
 }
}

Create & Publish Your Own Pulumi Components

Want to share your own best practices and cloud configurations with the Pulumi community? You can author a Pulumi Component and publish it to Pulumi Registry. We’ll have fully self-service publishing in 2022, but until then, we’re happy to work with you to publish your Component. You can get started using the link below.

👉 Build and publish a Pulumi Component

What’s Next for Pulumi Registry?

Today’s announcements are just the start of the journey for Pulumi Registry. In the coming months, we’ll continue to invest in making it even easier to author and publish a Pulumi Package. In 2022, we’ll also launch self-service publishing on Pulumi Registry, so you can share your packages directly with the Pulumi community. And, soon we’ll be bringing all of the goodness of Registry–the central index, the easy-to-use documentation, and more–to Pulumi Enterprise customers, so you can host Pulumi Packages private to your organization.

✅ Explore the Registry Now: Pulumi Registry

✅ Try Pulumi: Sign Up for Pulumi Cloud

Oct. 6 releases: AWS Native Provider, more refresh options

Alex Mullans — Wed, 06 Oct 2021 08:00:00 -0700

It’s been a busy few weeks at Pulumi, including for some of our community contributors! Read on to see what’s new.

New and updated cloud providers
Pulumi CLI and core technologies
- New project configuration option to refresh before all write operations

New and updated cloud providers

AWS Native Provider, powered by the AWS Cloud Control API

Last week, we announced the release of the new AWS Native provider for Pulumi, which is now available in preview. AWS is the most-used cloud provider across the Pulumi ecosystem, and with the new AWS Native provider, we are focused on delivering the best possible support for the AWS platform to all Pulumi users. The AWS Native provider offers same-day support for all new AWS features and releases covered by the newly released AWS Cloud Control API, which typically supports new AWS features on the day of launch.

This release also includes a new tool for migrating existing CloudFormation templates into Pulumi programs in your favorite language, powered by the new AWS Native provider and AWS Cloud Control API, as well as the ability to deploy any 3rd party resources in the CloudFormation Registry, including resources from Atlassian, Datadog, Densify, Dynatrace, Fortinet, New Relic, and Spot by NetApp.

Learn more in the Announcing AWS Native blog post

Using AWS Graviton2 processors in AWS Lambda functions

Last week, AWS launched the ability to use Graviton2 processors with AWS Lambda functions. Using Graviton2 processors gives you access to the improved price performance of that architecture. Pulumi’s AWS provider shipped with same-day support for this new functionality.

Learn more in the AWS Lambda functions blog post

New resources in the Azure Native provider

We shipped new versions of the Azure Native provider (1.29.0 through 1.34.0) that collectively added 14 new resources across services like Azure SQL, Azure API Management, and more that you can now manage with the Azure Native provider.

Pulumi CLI and core technologies

In this milestone, we shipped Pulumi versions 3.13.0 through 3.13.2. The full list of changes in each version is available in the changelog; read on to learn about some of the biggest changes.

New project configuration option to refresh before all write operations

Some Pulumi users have told us that they want to configure the behavior of their project such that a stack is always refreshed before any write operation (preview/up/destroy). This can help you detect and catch drift in your infrastructure, which can be introduced if someone makes changes to your infrastructure outside of the Pulumi program (e.g. in the cloud provider console).

Now, you can add the following option to your Pulumi.yaml project configuration file to always run a pulumi refresh before any write operation:

options:
 refresh: always

Learn more in this GitHub issue

Output property enhancements

To make Pulumi Component Package previews more useful and to improve parallelism of operations like destroy, we’ve expanded the definition of the Output property type to include dependencies, as well as its resolved value, and support to serialize the outputs.

Thanks @BenjaminSchiborr for their contribution in improving error messages related to these enhancements!

Sep. 16 releases: Helm Release, pulumi about, easier invites

Alex Mullans — Thu, 16 Sep 2021 08:00:00 -0700

It’s been a busy few weeks at Pulumi, including for some of our community contributors! Read on to see what’s new.

New and updated cloud providers
Pulumi CLI and core technologies
Pulumi Service and Pulumi.com
- Organization invites now send you to the correct identity provider

New and updated cloud providers

New Helm Release resource for Kubernetes

Pulumi’s existing Helm Chart resource provides support for deploying Helm charts via templating the chart and deploying it as Pulumi resources. This means that some features of Helm that are not available via Helm templating do not behave quite the same using the Pulumi Helm Chart resource as they do when deployed via the helm CLI itself - for example features like Helm lifecycle hooks and handling sub-charts and dependencies. As Helm and its usage evolved over the years, Pulumi users using the Chart resource have noted that in many cases they would like a resource that provides the exact same behave as the helm CLI, but can be orchestrated as part of a Pulumi deployment.

The new Helm Release resource provides Pulumi users more options to choose the right tool for their use-case. In contrast to the existing Helm Chart integration, which is implemented as a Component Resource that extracts the corresponding Kubernetes resources’ manifests from the chart and installs them as if they were individually specified in the Pulumi user program, the new Helm Release resource uses the Helm SDK directly to perform the deployment, providing the exact same behavior as a corresponding helm CLI deployment.

You can get started with the Helm Release resource in versions v3.7.0 and higher of the Pulumi Kubernetes Provider and SDK in all Pulumi supported languages.For example, in the following snippet we install Redis through a Helm Chart:

import * as random from "@pulumi/random";
import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

// Create a password
const pass = new random.RandomPassword("pass", {length:10});
const redisPassword = pass.result;

const namespace = new k8s.core.v1.Namespace("redis-ns");

const release = new k8s.helm.v3.Release("redis-helm", {
 chart: "redis",
 repositoryOpts: {
 repo: "https://charts.bitnami.com/bitnami",
 },
 version: "13.0.0",
 namespace: namespace.metadata.name,
 // Values from Chart's parameters specified hierarchically,
 // see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters for reference.
 values: {
 cluster: {
 enabled: true,
 slaveCount: 3,
 },
 metrics: {
 enabled: true,
 service: {
 annotations: {
 "prometheus.io/port": "9127",
 }
 },
 },
 global: {
 redis: {
 password: redisPassword,
 }
 },
 rbac: {
 create: true,
 }
 },
 // By default Release resource will wait till all created resources
 // are available. Set this to true to skip waiting on resources being
 // available.
 skipAwait: false,
});


// We can lookup resources once the release is installed. The release's
// status field is set once the installation completes, so this, combined
// with `skipAwait: false` above, will wait to retrieve the Redis master
// ClusterIP till all resources in the Chart are available.
const srv = k8s.core.v1.Service.get("redis-master-svc",
 pulumi.interpolate`${release.status.namespace}/${release.status.name}-redis-master`);
export const redisMasterClusterIP = srv.spec.clusterIP;

package main

import (
 "fmt"

 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/core/v1"
 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/helm/v3"
 "github.com/pulumi/pulumi-random/sdk/v4/go/random"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {

 namespace, err := corev1.NewNamespace(ctx, "redis-ns", nil)
 if err != nil {
 return err
 }

 redisPassword, err := random.NewRandomPassword(ctx, "pass", &random.RandomPasswordArgs{
 Length: pulumi.Int(10),
 })
 if err != nil {
 return err
 }

 rel, err := helm.NewRelease(ctx, "redis-helm", &helm.ReleaseArgs{
 Chart: pulumi.String("redis"),
 RepositoryOpts: helm.RepositoryOptsArgs{
 Repo: pulumi.String("https://charts.bitnami.com/bitnami"),
 },
 Version: pulumi.String("13.0.0"),
 Namespace: namespace.Metadata.Name(),

 // Values from Chart's parameters specified hierarchically,
 // see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters for reference.
 Values: pulumi.Map{
 "cluster": pulumi.Map{
 "enabled": pulumi.Bool(true),
 },
 "metrics": pulumi.Map{
 "enabled": pulumi.Bool(true),
 "service": pulumi.Map{
 "annotations": pulumi.StringMap{
 "prometheus.io/port": pulumi.String("9127"),
 },
 "type": pulumi.String("ClusterIP"),
 },
 },
 "global": pulumi.Map{
 "redis": pulumi.Map{
 "password": redisPassword.Result,
 },
 },
 "rbac": pulumi.BoolMap{
 "create": pulumi.Bool(true),
 },
 },

 // By default Release resource will wait till all created resources
 // are available. Set this to true to skip waiting on resources being
 // available.
 SkipAwait: pulumi.BoolPtr(false),
 })
 if err != nil {
 return err
 }

 // We can lookup resources once the release is installed. The release's
 // status field is set once the installation completes, so this, combined
 // with `skipAwait: pulumi.BoolPtr(false)` above, will wait to retrieve
 // the Redis master ClusterIP till all resources in the Chart are available.
 svc := pulumi.All(rel.Status.Namespace(), rel.Status.Name()).
 ApplyT(func(r interface{}) (interface{}, error) {
 arr := r.([]interface{})
 namespace := arr[0].(*string)
 name := arr[1].(*string)
 svc, err := corev1.GetService(ctx,
 "redis-master-svc",
 pulumi.ID(fmt.Sprintf("%s/%s-redis-master", *namespace, *name)),
 nil,
 )
 if err != nil {
 return "", nil
 }
 return svc.Spec.ClusterIP(), nil
 })
 ctx.Export("redisMasterClusterIP", svc)

 return nil
 })
}

import pulumi
from pulumi import Output
from pulumi_random.random_password import RandomPassword
from pulumi_kubernetes.core.v1 import Namespace, Service
from pulumi_kubernetes.helm.v3 import Release, ReleaseArgs, RepositoryOptsArgs

namespace = Namespace("redis-ns")

redis_password = RandomPassword("pass", length=10)

release_args = ReleaseArgs(
 chart="redis",
 repository_opts=RepositoryOptsArgs(
 repo="https://charts.bitnami.com/bitnami"
 ),
 version="13.0.0",
 namespace=namespace.metadata["name"],

 # Values from Chart's parameters specified hierarchically,
 # see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters
 # for reference.
 values={
 "cluster": {
 "enabled": True,
 "slaveCount": 3,
 },
 "metrics": {
 "enabled": True,
 "service": {
 "annotations": {
 "prometheus.io/port": "9127",
 }
 },
 },
 "global": {
 "redis": {
 "password": redis_password.result,
 }
 },
 "rbac": {
 "create": True,
 },
 },
 # By default Release resource will wait till all created resources
 # are available. Set this to true to skip waiting on resources being
 # available.
 skip_await=False)

release = Release("redis-helm", args=release_args)

# We can lookup resources once the release is installed. The release's
# status field is set once the installation completes, so this, combined
# with `skip_await=False` above, will wait to retrieve the Redis master
# ClusterIP till all resources in the Chart are available.
status = release.status
srv = Service.get("redis-master-svc",
 Output.concat(status.namespace, "/", status.name, "-redis-master"))
pulumi.export("redisMasterClusterIP", srv.spec.cluster_ip)

using Pulumi;
using System.Collections.Generic;
using Pulumi.Random;
using Pulumi.Kubernetes.Core.V1;
using Pulumi.Kubernetes.Types.Inputs.Helm.V3;
using Pulumi.Kubernetes.Helm.V3;

class MyStack : Stack
{
 public MyStack()
 {
 var ns = new Pulumi.Kubernetes.Core.V1.Namespace("redis-ns");
 var redisPassword = new RandomPassword("pass", new RandomPasswordArgs
 {
 Length = 10,
 });

 var releaseArgs = new ReleaseArgs
 {
 Chart = "redis",
 RepositoryOpts = new RepositoryOptsArgs
 {
 Repo = "https://charts.bitnami.com/bitnami",
 },
 Version = "13.0.0",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 // Values from Chart's parameters specified hierarchically,
 // see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters
 // for reference.
 Values = new InputMap<object>
 {
 ["cluster"] = new Dictionary<string,object>
 {
 ["enabled"] = true,
 ["slaveCount"] = 3,
 },
 ["metrics"] = new Dictionary<string,object>
 {
 ["enabled"] = true,
 ["service"] = new Dictionary<string, object>
 {
 ["annotations"] = new Dictionary<string, string>
 {
 ["prometheus.io/port"] = "9127",
 },
 }
 },
 ["global"] = new Dictionary<string,object>
 {
 ["redis"] = new Dictionary<string, object>
 {
 ["password"] = redisPassword.Result,
 },
 },
 ["rbac"] = new Dictionary<string,object>
 {
 ["create"] = true,
 }
 },
 // By default Release resource will wait till all created resources
 // are available. Set this to true to skip waiting on resources being
 // available.
 SkipAwait = false,
 };

 var release = new Release("redis-helm", releaseArgs);

 // We can lookup resources once the release is installed. The release's
 // status field is set once the installation completes, so this, combined
 // with `skip_await=False` above, will wait to retrieve the Redis master
 // ClusterIP till all resources in the Chart are available.
 var status = release.Status;
 var service = Service.Get("redist-master-svc", Output.All(status).Apply(
 s => $"{s[0].Namespace}/{s[0].Name}-redis-master"));
 this.RedisMasterClusterIP = service.Spec.Apply(spec => spec.ClusterIP);
 }

 [Output]
 public Output<string> RedisMasterClusterIP { get; set; }
}

Learn more in the announcement blog post

New providers: MinIO and Snowflake

You can now use Pulumi’s modern infrastructure-as-code solution to manage new resources! Our new provider for MinIO gives you access to the full range of high-performance, S3-compatible object storage offered by MinIO. Likewise, our new provider for Snowflake lets you create and manage many of the data engineering, data lake, data warehouse, and other data resources in Snowflake’s data cloud.

Learn more in the MinIO and Snowflake provider docs

New resources in the Azure Native provider

We shipped new versions of the Azure Native provider (1.22.0 through 1.28.0) that collectively added 15 new resources across services like SQL Server, Kusto, DocumentDB, and Power BI that you can now manage with the Azure Native provider.

See the full list

Pulumi CLI and core technologies

In this milestone, we shipped Pulumi versions 3.10.2 through 3.12.0. The full list of changes in each version is available in the changelog; read on to learn about some of the biggest changes.

New `pulumi about` command for easier troubleshooting

To make it easier to troubleshoot errors in the Pulumi CLI, we’ve added a new pulumi about command that collects useful information like the plugins you have installed, the OS and architecture you’re running, the backend that’s storing your state, your CLI version, and more.

Learn more in this GitHub issue

`pulumi config set` hides secret values

Previously, setting a config value using pulumi config set would write the value to stdout, which could be undesirable if the config value was a secret. Now, config values that look like secrets will be hidden and the config key name will be written to stdout instead.

Thanks @JasonWhall for contributing this change!

Learn more in this GitHub PR

Dynamic providers in Python can now given a custom name

When building a dynamic provider, it can be useful to give the provider a name so that at runtime, you can identify which provider is actually making a change. Previously, it wasn’t possible to do so in Python. Now, you can pass a name parameter to the dynamic provider’s constructor:

class CustomResource(
 Resource, module="custom-provider", name="CustomResource"
):
 ...

Thanks @jancespivo for contributing this change!

Learn more in this GitHub issue

Automation API in .NET can use plugin installation options `exact` and `server`

When running Pulumi interactively, you can use the pulumi plugin install command to manually install plugins required by your program, possibly from a server where you host your own plugins. You can now do the same task in the .NET Automation API:

await workspace.InstallPluginAsync("myplugin", "v0.0.1", options: new PluginInstallOptions
{
 ServerUrl = "https://<custom-url>",
});

Thanks @orionstudt for contributing this change!

Learn more in this GitHub PR

Schema checker for Pulumi Packages

When authoring a Pulumi Package, it can be helpful to validate that your schema is correct. Now, you can validate your schema by running the pulumi schema check command.

Learn more in this GitHub PR

Pulumi Service and Pulumi.com

Organization invites now send you to the correct identity provider

Previously, when you invited a collaborator to your organization, that collaborator needed to know which identity provider (email, GitHub, GitLab, Bitbucket, or SAML SSO) that your organization uses in order to sign up correctly and get access to the organization. Now, any invites you send will direct your new collaborator to the correct identity provider automatically.

August 18 releases: EKS, Pulumi Component methods, dependsOn

Alex Mullans — Wed, 18 Aug 2021 08:00:00 -0700

Summer is quickly flying by, and we’ve been hard at work with more new features and updates across the Pulumi ecosystem!

New and updated cloud providers
- EKS Pulumi Package supports more options for managed node groups
- New resources in the Azure Native provider
Pulumi CLI and core technologies
Pulumi Service and Pulumi.com
- Azure Native resources now link to the Azure Portal

New and updated cloud providers

EKS Pulumi Package supports more options for managed node groups

As the AWS team has improved the Elastic Kubernetes Service (EKS), they’ve added additional options, particularly for managed node groups, that haven’t been available in the EKS Pulumi Package. We’ve updated the package to add these missing options.

Learn more in these GitHub issues 1, 2, 3

New resources in the Azure Native provider

We shipped 2 new versions of the Azure Native provider (1.20.0 through 1.21.0) that collectively added 8 new resources that you can manage with the Azure Native provider, including core networking and IoT security resources.

See the full list

Pulumi CLI and core technologies

In this milestone, we shipped Pulumi versions 3.9.0 through 3.10.1. The full list of changes in each version is available in the changelog; read on to learn about some of the biggest changes.

Methods for Pulumi Component Packages

Pulumi Components often have methods that provide additional functionality associated with an instance of a component. Previously, when authoring a Pulumi Component Package, you couldn’t make a component’s methods available in all languages. Now, with updates to the Pulumi Packages schema and Provider interface, you can! We’ll have a dedicated blog post out soon with more details on how to use methods in your Pulumi Component Package.

Learn more in this GitHub issue

`dependsOn` now works for Pulumi Component Packages, and more

The dependsOn option creates a list of explicit dependencies between resources, which can be useful when you need to explicitly specify additional dependencies that Pulumi doesn’t know about but must still respect. Previously, dependsOn worked differently depending on the language your Pulumi code was written in or when referencing a component inside a Pulumi Component Package. Now, depending on a Pulumi Component Package’s component will always wait on all of that component’s children, regardless of language. Additionally, you can now pass Inputs and Outputs to dependsOn, which can be especially useful in Kubernetes and Helm workflows where you need to depend on the output of a resource created by a Helm chart.

To see this work in action, check out the Staged App Rollout Gated by Prometheus Checks example, where we create a staged rollout (from 3-replica canary -> 10-replica staging) gated by confirming that the P90 response time reported by Prometheus is less than some amount. The relevant code is the last line of this snippet, which lets your Kubernetes deployment depend on a Prometheus resource:

const p8sService = prometheus.getResource("v1/Service", "p8s-prometheus-server");
const p8sDeployment = prometheus.getResource(
 "extensions/v1beta1/Deployment",
 "p8s-prometheus-server",
);

const localPort = 9090;
const forwarderHandle = util.forwardPrometheusService(p8sService, p8sDeployment, {
 localPort,
});

// Canary ring. Replicate instrumented Pod 3 times.
const canary = new k8s.apps.v1beta1.Deployment(
 "canary-example-app",
 { spec: { replicas: 1, template: instrumentedPod } },
 { dependsOn: p8sDeployment },
);

Learn more in this GitHub issue and this issue

Aliases can now be used with providers

Pulumi has historically enabled you to alias resources, which can help with scenarios where you want to change a resource’s name without causing a delete and replace of that resource. Now, you can also alias providers, which can make it easier to refactor stacks and code over time. In the example below, an AWS provider instance is created with the alias “newName”:

import * as aws from "@pulumi/aws";

const providerName = "newName";
const awsProvider = new aws.Provider(providerName, {
 region: "us-west-2",
}, { aliases: [{ name: "oldName" }]});

const bucket = new aws.s3.Bucket("my-bucket", {}, { provider: awsProvider });

export const bucketName = bucket.id;

Learn more in this GitHub issue

Pulumi Service and Pulumi.com

Azure Native resources now link to the Azure Portal

For popular cloud providers like Pulumi AWS and Pulumi Azure Classic, the Pulumi Service contains convenient links from the Resources tab directly to the resource’s management page in the cloud provider console. Now, resources you manage using the Pulumi Azure Native Provider provide these convenient links to the Azure Portal.

July 28 releases: K8s GitOps, autonaming in Google Native

Alex Mullans — Wed, 28 Jul 2021 08:00:00 -0700

Another great iteration has just wrapped up, so it’s time for another edition of the Pulumi release notes! We’ve been busy smashing bugs across our products (over 100!), but we’ve also got several new updates across Pulumi providers, the CLI, and the Pulumi Service:

New and updated cloud providers
Pulumi CLI and core technologies
- Automation API: Select your backend
- Better handling of subtypes in Python
Pulumi Service and Pulumi.com
- Get started with Pulumi, directly from the Pulumi Service

New and updated cloud providers

Pulumi Kubernetes Operator supports GitOps workflow

For some Kubernetes workflows, developers want to be able to store their configuration in a Git repository but ensure that their cluster automatically stays up-to-date with the latest configuration when it’s checked in. With the latest update to the Pulumi Kubernetes Operator, you can set up this type of GitOps workflow. To get started, specify the fully-qualified branch name that you want to track in the Stack CustomResource (e.g., refs/heads/main will track the main branch). The operator will track the specified branch and automatically deploy new commits that are pushed to the branch. For example, the Kubernetes YAML below will tell the Pulumi Kubernetes Operator to poll the production branch of the target GitHub repo, and deploy an update to the acmecorp/s3-op-project/production stack when there are changes.

apiVersion: pulumi.com/v1alpha1
kind: Stack
metadata:
 name: s3-bucket-stack
spec:
 stack: acmecorp/s3-op-project/production
 projectRepo: https://github.com/acmecorp/test-s3-op-project
 branch: refs/heads/production
 config:
 aws:region: us-east-2

Learn more in this GitHub issue

12 new resources in the Azure Native provider

We shipped 4 new versions of the Azure Native provider (1.16.0 through 1.19.0) that collectively added 12 new resources that you can manage with the Azure Native provider, including Fast Healthcare Interoperability Resources (FHIR) and Azure Relay resources.

See the full list

Google Native: autonaming; easier management of project and region settings

The preview of the Google Native provider continues to evolve. We shipped a new version of the Google Native provider (0.5.0) that includes:

Full auto-naming support, just like our other providers
An easier way to use arguments like machineType that expect a fully-qualified string like projects/${project}/zones/${zone}/machineTypes/f1-micro: you can now use just the last value (f1-micro) in your program and the provider will construct the fully-qualified string
Automatic selection of the project, region, and zone based on your configuration, so you no longer need to pass these values to the SDK’s constructor
An improved implementation of BucketObject

The screenshot below, from the webserver-ts example, shows the auto-naming, automatic project and region selection, and improved fully-qualified string settings:

See the full list of updates

Pulumi CLI and core technologies

In this milestone, we shipped Pulumi versions 3.7.0, 3.7.1, and 3.8.0. The full list of changes in each version is available in the linked changelog; read on to learn about some of the biggest changes.

Automation API: Select your backend

Previously, the Automation API only used the backend that you selected the last time you ran the pulumi login command, which led to challenges for developers working across multiple backends for different projects. Now, you can set the PULUMI_BACKEND_URL variable in the environment where you run your Automation API program and it will use the backend specified by that URL.

Learn more in this GitHub issue

Better handling of subtypes in Python

Subtypes now correctly propagate through a call to Output[T]. For example, if Cow is a subtype of Animal, this commit makes it so Output[Cow] is treated as a subtype of Output[Animal]. Before this change, users of the Python SDK would occasionally need to use an .apply(lambda x: x) command when working with subtypes.

Learn more in this GitHub issue

Pulumi Service and Pulumi.com

Get started with Pulumi, directly from the Pulumi Service

We’re always looking for ways to make it easier for people who are new to Pulumi to get started successfully. This release, we’ve added a new dashboard card to your individual user dashboard on the Pulumi Service that helps you set up the Pulumi CLI and deploy your first stack of cloud infrastructure. Look for the “Get Started” card on your dashboard on the Pulumi Service.

July 7 releases: new pricing, replaceOnChanges, and more

Alex Mullans — Wed, 07 Jul 2021 08:00:00 -0700

In this update:

New and updated cloud providers
Pulumi CLI and core technologies
Pulumi Service and Pulumi.com
- New usage-based pricing for your whole team

New and updated cloud providers

Pulumi supports Confluent Cloud

The new Confluent Cloud provider for Pulumi allows you to manage cloud-native Apache Kafka services with Pulumi infrastructure-as-code.

const ccloud = require("@pulumi/confluent")

const env = new ccloud.ConfluentEnvironment("ts-environment");

import * as confluent from "@pulumi/confluent";

const env = new ccloud.ConfluentEnvironment("ts-environment");

import pulumi_confluent as confluent

environment = ccloud.ConfluentEnvironment("py-env")

package main

import (
 "github.com/pulumi/pulumi-confluent/sdk/go/confluent"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {

 env, err := confluent.NewConfluentEnvironment(ctx, "py-env", nil)
 if err != nil {
 return err
 }

 return nil
 })
}

using System.Collections.Generic;
using System.Threading.Tasks;
using Pulumi;
using Pulumi.Confluent;

class Program
{
 static Task Main() =>
 Deployment.Run(() => {
 var environment = new CCloud.ConfluentEnvironment("csharp-env");
 });
}

Get started in the Confluent Cloud provider docs.

GitHub Actions guide in all languages

Previously, the GitHub Actions user guide only provided documentation for Node.js projects. We’ve now updated it to include all Pulumi languages.

name: Pulumi
on:
 - pull_request
jobs:
 preview:
 name: Preview
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v2
 - uses: actions/setup-node@v2
 with:
 node-version: 14.x
 - name: Configure AWS Credentials
 uses: aws-actions/configure-aws-credentials@v1
 with:
 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
 aws-region: ${{ secrets.AWS_REGION }}
 aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
 - run: npm install
 - uses: pulumi/actions@v3
 with:
 command: preview
 stack-name: dev
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

name: Pulumi
on:
 - pull_request
jobs:
 preview:
 name: Preview
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v2
 - uses: actions/setup-python@v2
 with:
 python-version: 3.7
 - name: Configure AWS Credentials
 uses: aws-actions/configure-aws-credentials@v1
 with:
 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
 aws-region: ${{ secrets.AWS_REGION }}
 aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
 - run: pip install -r requirements.txt
 - uses: pulumi/actions@v3
 with:
 command: preview
 stack-name: dev
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

name: Pulumi
on:
 - pull_request
jobs:
 preview:
 name: Preview
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v2
 - uses: actions/setup-go@v1
 with:
 go-version: 1.17.x
 - name: Configure AWS Credentials
 uses: aws-actions/configure-aws-credentials@v1
 with:
 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
 aws-region: ${{ secrets.AWS_REGION }}
 aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
 - run: go mod download
 - uses: pulumi/actions@v3
 with:
 command: preview
 stack-name: dev
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

name: Pulumi
on:
 - pull_request
jobs:
 preview:
 name: Preview
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v2
 - uses: actions/setup-dotnet@v1
 with:
 dotnet-version: 3.1.x
 - name: Configure AWS Credentials
 uses: aws-actions/configure-aws-credentials@v1
 with:
 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
 aws-region: ${{ secrets.AWS_REGION }}
 aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
 - uses: pulumi/actions@v3
 with:
 command: preview
 stack-name: dev
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

Learn more in the GitHub Actions User Guide

Kubernetes: new `skipAwait` option

For some Kubernetes use cases, including Helm charts, you may want to be able to turn off all await logic, so that you can use Kubernetes via Pulumi in a way that feels more similar to the default Kubernetes API and kubectl experience. Previously, you could achieve this by marking individual entities with a custom annotation to skip the await logic. Now, you can use the new skipAwait option on Helm charts and YAML components to turn off the await logic for those charts/components and all of their child resources. Please note that skipAwait changes some default Pulumi behaviors: outputs are not populated and incorrect deployments aren’t identified. However, in cases where you don’t need the result of the chart/component, you can speed up deployments using skipAwait.

Learn more at pulumi/pulumi-kubernetes#1421

10 new resources in the Azure Native provider

We shipped 5 new versions of the Azure Native provider (1.11.0 through 1.15.0) that collectively added 10 new resources that you can manage with the Azure Native provider, including Azure Arc data services and Azure Event Grid resources.

See the full list

Google Native provider adds support for enums and more

The preview of the Google Native provider continues to evolve. We shipped 2 new versions of the Google Native provider (0.3.0 and 0.4.0) that include:

Refined, shorter resource names that are easier to type and easier to find
Enum support for all properties exposed as enums in the underlying Google Discovery Documents

See the full list of updates

Pulumi CLI and core technologies

In this milestone, we shipped Pulumi versions 3.5.1 and 3.6.0. The full list of changes in each version is available in the linked changelog; read on to learn about some of the biggest changes.

New `replaceOnChanges` resource option

Sometimes, you may want to force the replacement of a resource even if the changes you’ve made to that resource don’t indicate that they require a replacement. One example is Kubernetes CustomResourceDefinitions: many CRDs can’t be updated in place due to forward-incompatibilities, so you may need to force a replacement. Previously, you could do this by changing the physical name of the resource, but there are cases where it is important to keep the physical name the same.

You can use the new replaceOnChange resource option to force resource replacement when the specified properties are changed. This can be combined with the existing deleteBeforeReplace resource option to force a resource to be deleted before replacement. For example, you can specify that an AWS S3 Bucket resource be replaced every time its index document changes:

const bucket = new aws.s3.Bucket("my-bucket", {
 website: {
 indexDocument: "index2.html",
 },
}, { replaceOnChanges: ["website.indexDocument"] });

Learn more at pulumi/pulumi#7226

`pulumi watch` now supports a `--path` option

Previously, the pulumi watch command always watched the entire Pulumi project directory (recursively). This caused duplicate updates for developers working in monorepos or developers with application code and infrastructure code in the same directory. Now, you can add a --path option to the pulumi watch command and specify the exact paths to watch.

pulumi watch --path path/to/pulumi/code/

Learn more at pulumi/pulumi#6620

Improved Python performance

We’ve improved the performance of common Pulumi commands like up and preview for Python programs by removing unnecessary logging and version checks.

Learn more at pulumi/pulumi#7291 and pulumi/pulumi#7292

Pulumi Service and Pulumi.com

New usage-based pricing for your whole team

We launched Pulumi Team Edition, a new way for teams to adopt and use Pulumi and the Pulumi Service to collaborate on building, managing, and deploying cloud infrastructure as code. Pulumi Team Edition is priced based on the number of cloud resources under management, with a generous free tier to ensure that teams can get up and running with Pulumi Team Edition at no cost.

Learn more in the pricing announcement blog post

June 10 release notes: AWS ECS Anywhere, Sumo Logic, & more!

Alex Mullans — Thu, 10 Jun 2021 09:00:00 -0700

We regularly ship updates across the Pulumi ecosystem, with a release of the Pulumi CLI every two weeks, frequent releases of our cloud providers in the Registry, and regular updates to the Pulumi Service. You can read about most of those changes in the CHANGELOG.md files in each open source repository. To highlight some of the most exciting releases—including, for the first time, updates to the Pulumi Service—we’re trying something new: a regular release notes post for Pulumi release news, features, and updates. If you find it useful, or have ideas on how we could improve it, let us know on Twitter or in the Pulumi Community Slack!

In this update:

New and updated cloud providers
Pulumi engine and core technologies
- Input parameters can now be marked as secret
- Pulumi Packages: schema documentation and enhancements
Pulumi Service and Pulumi.com
- Easier acceptance of invites to organizations
- Self-Hosted Pulumi Service now supports Azure Services and Minio storage

New and updated cloud providers

Pulumi supports Amazon ECS Anywhere and Amazon App Runner

Earlier this month, Amazon announced ECS Anywhere, an extension of Amazon Elastic Container Service that allows you to deploy containers to almost any server, and AWS App Runner, a new fully-managed container application service. Pulumi had full same-day support for both new services.

Learn how to get started with ECS Anywhere and App Runner.

Pulumi supports Sumo Logic

The new Sumo Logic provider for Pulumi allows you to manage Sumo Logic’s big data and analytics services with Pulumi infrastructure-as-code.

const sumologic = require("@pulumi/sumologic")

const folder = new sumologic.Folder("folder", {
 description: "A test folder",
 parentId: "<personal folder id>",
});

import * as sumologic from "@pulumi/sumologic";

const folder = new sumologic.Folder("folder", {
 description: "A test folder",
 parentId: "<personal folder id>",
});

import pulumi_sumologic as sumologic

folder = sumologic.Folder("folder",
 description="A test folder",
 parent_id="<personal folder id>")

import (
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
 "github.com/pulumi/pulumi-sumologic/sdk/go/sumologic"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := sumologic.NewFolder(ctx, "folder", &sumologic.FolderArgs{
 Description: pulumi.String("A test folder"),
 ParentId: pulumi.String("<personal folder id>"),
 })
 if err != nil {
 return err
 }

 return nil
 })
}

using System.Collections.Generic;
using System.Threading.Tasks;
using Pulumi;
using Pulumi.SumoLogic;

class Program
{
 static Task Main() =>
 Deployment.Run(() => {
 var folder = new SumoLogic.Folder("folder", new SumoLogic.FolderArgs
 {
 Description = "A test folder",
 ParentId = "<personal folder id>",
 });
 });
}

Get started in the SumoLogic provider docs.

100 new resources in the Azure Native provider

We shipped 3 new versions of the Azure Native provider (1.10.0, 1.9.0, and 1.8.0) that collectively added 100 new resources that you can manage with the Azure Native provider. These new provider versions give you full access to the new resources and services announced at Microsoft Build 2021, including new analytics, machine learning, and security capabilities across the Azure platform.

See the full list

Pulumi CLI and core technologies

In this milestone, we shipped Pulumi versions 3.3.0, 3.3.1, and 3.4.0. The full list of changes in each version is available in the linked changelog; read on to learn about some of the biggest changes.

Input parameters can now be marked as secret

It’s now possible to mark any input parameter as a secret, which can be helpful when you need to pass in secret data, like when creating a Kubernetes Secret resource. Pulumi will protect these input parameters by encrypting them in the state and obfuscating them in outputs.

Learn more at pulumi/pulumi#7062

Pulumi Packages: schema documentation and enhancements

Pulumi Packages, released at PulumiUP earlier this year enable cloud infrastructure resource provisioning to be defined once, in your language of choice, and made available to users in all Pulumi languages. A key part of Pulumi Packages is the Pulumi schema that defines each resource in a package. We have now introduced new documentation for Pulumi Package Schema.

We’ve also updated the Pulumi Package schema to make it more powerful. For more complex Pulumi Package scenarios, you may want to be able to reference the types in one Pulumi Package from another Pulumi Package. Previously, this was only possible if you authored your Pulumi Package in Go; now, it’s possible in all Pulumi Languages.

Learn more at pulumi/pulumi#6777

Pulumi Service and Pulumi.com

Easier acceptance of invites to organizations

Previously, you could accept an invitation to join an organization even if you didn’t have the required identity provider (say, a SAML identity) associated with your Pulumi account. This left you unable to access the organization to which you just accepted an invite. Now, the Pulumi Service will prompt you to connect the required identity provider to your existing Pulumi account so you can successfully accept the invitation and join the organization.

Self-Hosted Pulumi Service now supports Azure services and Minio storage

The Self-Hosted Pulumi Service, which brings the full power of the Pulumi Service into your own environment, can now be deployed in a few new ways:

In Microsoft Azure, using Azure Blob Storage for state storage and Azure Key Vault for secret storage
Using Minio for state storage

Learn more about how to configure these options in the Pulumi API setup guide.

Model and program the cloud with Pulumi native providers

Alex Mullans — Tue, 20 Apr 2021 00:00:05 -0700

Pulumi native providers are a new type of Pulumi Package that give you the most complete and consistent interface for the modern cloud. Pulumi native providers bring the full power of the top cloud providers to the Pulumi Cloud Engineering Platform, faster and with more complete coverage than any other infrastructure as code offering. Today at PulumiUP, we announced native providers for Microsoft Azure (GA), Google Cloud (public preview), and AWS (later this year). Along with an existing native provider for Kubernetes, these providers enable you to build, deploy, and manage cloud infrastructure and applications for the most common cloud vendors and technologies. This best-in-class support for the major clouds joins our library of more than 50 cloud providers in the Registry and delivers on our promise of cloud engineering for any cloud, any architecture, and any language.

100% API coverage, same-day access

Legacy infrastructure as code tools rely on providers that are often hand-coded. These providers have fairly broad coverage of the modern cloud ecosystem, but they can’t always keep up with the pace of innovation of that ecosystem. That can leave you stuck with an awkward choice: wait until the provider is updated and miss out on the first-mover advantage, or try out the new infrastructure directly without any of the control, repeatability, or scalability you expect from infrastructure as code.

With Pulumi native providers, you don’t need to make that choice. Native providers are built directly and automatically from the cloud provider’s API. As a result, you can trust that these new native providers give you access to 100% of the resources covered by the cloud provider’s API and same-day access to feature updates and new services. When you see a new resource at AWS re:Invent, Microsoft Build, or Google Cloud Next, you can be sure that it’ll be available for use in Pulumi the same day that it’s added to the provider’s API. You can get productive with that new resource quickly because native providers also pull documentation from the API. As with all Pulumi providers, you get the full benefits of code completion, syntax highlighting, and everything else you expect from your programming language and development environment. That means you can start using those new services as soon as you hear about them without losing any of the benefits and protections of your infrastructure as code. And of course, like Pulumi itself, Pulumi native providers are open source, so you can see the code, file issues, and contribute fixes and new features.

Azure Native is generally available

Microsoft delivers new value to the Azure platform quickly and frequently, with hundreds of new resources, features, and improvements just this year. As those updates became available, we added them to the Pulumi Azure Native provider. Since we announced Azure Native last September, we’ve maintained its 100% coverage of the Azure Resource Manager API surface by adding the 166 new resources (e.g. new service or features within existing services) added to the ARM APIs. Those new resources have been contained in the 279 daily builds of the provider we’ve released since that announcement. These releases guarantee same-day coverage of Azure updates, so you don’t have to wait to use new Azure features and updates.

Since the release of Azure Native, many Azure Classic provider users have already started using it in their projects and filing bugs and feature requests. We’ve been fixing those issues, and today we’re excited to make Azure Native generally available at version 1.0.0. Azure Native is your best choice for any new project that uses resources from Azure; you can also mix in new resources from Azure Native in a project that uses the Azure Classic provider today. If you’d like, you can migrate existing projects to use Azure Native using the migration guide. However, the Azure Classic provider remains fully supported.

A great example of the unique value you get from the Pulumi native provider approach is Consumption Budgets. Consumption Budgets allow you to control your Azure spend proactively, so you’re not surprised when your bill arrives. Consumption Budgets are covered by the Azure Resource Manager API, so you can create a Consumption Budget with Azure Native right now. In contrast, Consumption Budgets has been a pending feature request for over 2 years in the Terraform provider used by the Azure Classic provider.

With the Azure platform evolving as rapidly as it does, you might worry that it’ll be hard to manage API versions or know which APIs are stable and ready to use. But fear not! Each major version of the Azure Native provider exposes a top-level namespace with a consistent set of stable APIs. Any breaking changes made in the Azure APIs will be released in a new major version of the Azure Native provider. If you need a different version of an API, you have access to that as well: every API version is exposed as a separate namespace in the provider.

👉 Get started with Azure Native

Public preview: a native provider for Google Cloud Platform

Google Cloud Platform is also growing at a rapid pace, with new features and updates almost every day. To track that growth, today at PulumiUP, we announced a public preview of GCP-Native, a Pulumi native provider for the Google Cloud Platform. Just like the Azure Native provider, GCP-Native gives you 100% coverage of every Google Cloud Platform API, so you can do things that aren’t possible with the current Terraform-based Google Cloud provider. And just like the Azure Native provider, Google Native is updated with a new release every time those APIs are updated, so you have access to new GCP features as soon as they’re released.

👉 Read the GCP-Native public preview announcement

AWS later this year, and more to come

Of course, we’re not stopping with native providers for Azure, Google Cloud, and Kubernetes. Later this year, we’ll release a native provider for Amazon Web Services. Just like the Azure Native and Google Cloud Native providers, the AWS Native provider will support 100% of CloudFormation resources and give you same-day access when new resources are released.

Pulumi native providers are the best way to build, deploy, and manage resources in AWS, Azure, GCP, and Kubernetes, but you’re not limited to just those clouds. Pulumi’s broad library of more than 50 cloud providers in the Registry remains available and ready to use.

You can help us decide which providers to build next! Add your vote to existing provider requests here or request a provider.

👉 See all the cloud providers you can use with Pulumi

A peek under the hood

Native providers fulfill a core promise of the Pulumi Cloud Engineering Platform: to be your most complete and consistent interface to model and program the cloud. Here’s how we deliver on that promise of 100% API coverage and same-day updates.

Full API coverage via automated nightly updates

Automation in GitHub Actions checks the Azure, Google, and Amazon API specifications every night and automatically generates a pull request to update the corresponding native provider Pulumi Package with any changes. That pull request updates the Pulumi Package’s schema (the metadata file that defines the resources, inputs, and outputs that make up the native provider), re-generates all four language SDKs, and creates a new minor version of the provider. That new minor version contains:

New namespaces for any new API versions, so you can access new features, even if they’re behind a preview API version
All non-breaking updates to the top-level APIs, so you get bug fixes and non-breaking improvements as soon as they’re ready

Periodically, we also create a new major version release of the provider that promotes new API major versions to the top-level namespace. This ensures that new APIs become part of the “default” set available in the top-level namespace.

If you want to dive deeper into how we keep the native provider Pulumi Packages up to date, you can see Azure Native’s automation in GitHub Actions and an example pull request.

Going beyond the API

Sometimes, a cloud provider doesn’t cover some scenarios or resources in their primary API. For example, the Azure Resource Manager API doesn’t currently include support for managing Azure Key Vault secrets and keys. In these cases, to provide the best experience, we hand-author extensions to the native provider Pulumi Package to cover these scenarios.

Keep exploring

These native provider announcements are just one part of Pulumi’s Cloud Engineering Platform. Watch the PulumiUP event for more.

👉 PulumiUP

Google Cloud Native provider: full coverage of Google Cloud

Alex Mullans — Tue, 20 Apr 2021 00:00:04 -0700

Today at PulumiUP, we announced the public preview of our Google Cloud Native provider. Google Cloud usage amongst Pulumi users has grown rapidly in the past year and Google Cloud has been adding new features and updates almost every day. The new native provider gives Pulumi users the best support possible for Google Cloud in Pulumi.

The Google Cloud Native provider works directly with the Google Cloud APIs, as surfaced by the Google API Discovery Service, instead of depending on a handwritten layer like other current infrastructure as code technologies. This new approach is similar to that which we’ve taken for our Kubernetes and Azure Native providers and ensures higher quality and complete coverage of the resources available in GCP.

Full API coverage

Google Cloud Native covers 100% of the resources available in the Google Cloud APIs. The new provider supports 508 resource types at launch, 10% more than the number supported by the previous Google Cloud provider, and over 70% more properties, each of which represents a new capability or feature exposed by Google Cloud but unavailable in other infrastructure as code tools. Every property of each resource is represented in the SDKs. In addition, every version of each resource is available, providing access to each version of beta and GA features in a stable way from a single provider.

Like our other native providers, the new SDKs include 100% coverage of Google Cloud resources.

Always up-to-date

Unlike the original Google Cloud provider, which requires manual work to keep updated, the new provider is always up-to-date with additions and changes to the Google Cloud APIs. This also ensures very high quality since a human doesn’t need to manually implement support for all of those services and features.

We generate Pulumi SDKs for Google Cloud Native automatically using updates from the Google API Discovery Service. This has been designed to enable an automated pipeline to release updated resources within hours after any current API specifications are merged. That pipeline will publish daily updates via automated builds and minor SDK versions every two weeks.

By generating the SDKs directly from the Google Cloud APIs, the Google Cloud Native provider is robust and reliable, with fewer moving parts involved and fewer sources of potential bugs and incompatibilities. For example, when Google’s new GKE Autopilot feature shipped, early builds of this provider supported it within hours, compared to over a month for legacy infrastructure as code tools.

Excited about a new service announced by Google? Chances are it’s already in the latest Google Cloud Native package!

Every language

Google Cloud Native is available in public preview today for all Pulumi languages. Because it’s built on the Pulumi Packages technology, every update to Google Cloud Native is available in all languages on the day of release. The new Google Cloud Native SDKs are open source on GitHub and available on npm, NuGet.org, PyPI, and Go modules.

import * as pulumi from "@pulumi/pulumi";
import * as google from "@pulumi/google-native";

const config = new pulumi.Config("google-native");
const project = config.require("project");

const region = "us-central1";
const clusterName = "gke-native";

const cluster = new google.container.v1.Cluster("cluster", {
 projectsId: project,
 locationsId: region,
 clustersId: clusterName,
 name: clusterName,
 parent: `projects/${project}/locations/${region}`,
 autopilot: { enabled: true }
});

import pulumi
from pulumi_google_native.container import v1 as container

config = pulumi.Config('google-native')
project = config.require('project')

region = "us-central1"
cluster_name = "gke-native"

cluster = container.Cluster(
 "cluster",
 projects_id=project,
 locations_id=region,
 clusters_id=cluster_name,
 name=cluster_name,
 autopilot=container.AutopilotArgs(enabled=True))

package main

import (
 container "github.com/pulumi/pulumi-google-native/sdk/go/google/container/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi/config"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 const region = "us-central1"
 const clusterName = "gke-native"

 pulumi.Run(func(ctx *pulumi.Context) error {
 conf := config.New(ctx, "google-native")
 project := conf.Require("project")
 cluster, err := container.NewCluster(ctx, "cluster", &container.ClusterArgs{
 Name: pulumi.StringPtr(clusterName),
 ClustersId: pulumi.String(clusterName),
 LocationsId: pulumi.String(region),
 ProjectsId: pulumi.String(project),
 Autopilot: container.AutopilotArgs{
 Enabled: pulumi.Bool(true),
 },
 })
 if err != nil {
 return err
 }

 ctx.Export("clusterSelfLink", cluster.SelfLink)

 return nil
 })
}

using System.Threading.Tasks;
using Pulumi;
using Pulumi.GoogleNative.Container.V1;
using Pulumi.GoogleNative.Container.V1.Inputs;

class Program
{
 static Task<int> Main() => Deployment.RunAsync(() =>
 {
 var config = new Config("google-native");
 var project = config.Require("project");

 var region = "us-central1";
 var clusterName = "gke-native";

 var cluster = new Cluster("cluster", new ClusterArgs
 {
 ProjectsId = project,
 LocationsId = region,
 ClustersId = clusterName,
 Name = clusterName,
 Autopilot = new AutopilotArgs { Enabled = true }
 });
 });
}

API documentation is available in our API reference.

Works side-by-side with the existing Pulumi GCP Provider

We will continue to invest in the existing Google Cloud provider. Pulumi users can use either provider or use both providers side-by-side in their applications.

A single Pulumi project can use both providers. The outputs of a resource from one provider can flow to inputs of a resource from the other provider. In both cases, they are the values available from Google Cloud itself, such as the name or ID of a Google Cloud resource.

If you have existing projects using the current Pulumi GCP provider, you can continue to use that provider, and Pulumi will continue to update it indefinitely. You can add new cloud resources using the existing provider or start creating new infrastructure with the new provider. At a later date in the preview, we will publish a migration guide to help you move existing projects from GCP to Google Cloud Native.

New Pulumi projects for Google Cloud will default to using Google Cloud Native once it reaches general availability later this year.

Getting started

New Pulumi templates are available for the Google Cloud Native provider:

$ pulumi new google-native-typescript

$ pulumi new google-native-python

$ pulumi new google-native-csharp

$ pulumi new google-native-go

Several more extensive examples are also available in the Pulumi Examples repo:

Containerized Ruby on Rails App Delivery using the Google Native Pulumi Provider: TypeScript
Google Cloud Functions: TypeScript

The preview of the Google Cloud Native provider is ready for you to try. It’s easy to get started by trying one of our examples. We look forward to hearing your feedback! We look forward to hearing your feedback! Please file any bugs and feature requests on GitHub or ask questions in the Pulumi Community Slack. We are eager to hear your feedback and how we can help.

Keep exploring

The Google Cloud Native announcement is just one part of Pulumi’s Cloud Engineering Platform. Watch the PulumiUP event for more.

👉 PulumiUP 👉 Google Cloud Native API reference

Introducing Pulumi Packages and multi-language Components

Alex Mullans — Tue, 20 Apr 2021 00:00:03 -0700

Pulumi Packages are the core technology that enables cloud infrastructure resource provisioning to be defined once, in your language of choice, and made available to users in all Pulumi languages. If you’ve used a Pulumi cloud provider, including one of our new Pulumi native providers, you’ve used a Pulumi Package. But until today, Pulumi Packages only worked with Pulumi Resources: direct, low-level representations of individual cloud services like object storage. Many of us, however, enjoy creating Pulumi Components, which combine low-level resources into higher-level, more opinionated building blocks like the production-grade Kubernetes cluster component in Pulumi EKS. Unfortunately, those components, though powerful and unique to Pulumi’s IaC approach, were previously confined to a single language: so if your infrastructure team built a component in Python, your developers who might want to use TypeScript could not use it.

In Pulumi 3.0 Pulumi Components can now be authored in a fundamentally multi-language way, enabling you to write them once in your preferred language and make them available in all the other languages supported by Pulumi—all thanks to Pulumi Packages.

👉 Read on to learn more about Pulumi Packages, or jump right in and author your own with the user guide

Using Pulumi Packages

Pulumi Packages bring the full power of the modern cloud and the amazing creativity of the Pulumi Community to you. Using a Pulumi Package is easy: find one, install it into your project, and get started.

Choose from any of Pulumi’s cloud provider packages in the Registry, or search the npm registry for other Pulumi Packages
Install into your project with npm i [package-name]
Reference a namespace in the newly-installed package and start writing code

Choose from any of Pulumi’s cloud provider packages in the Registry, or search PyPI for other Pulumi Packages
Install into your project with pip install [package-name]
Reference a namespace in the newly-installed package and start writing code

Choose from any of Pulumi’s cloud provider packages in the Registry, or search the NuGet Gallery for other Pulumi Packages
Install into your project with dotnet add package [package-name]
Reference a namespace in the newly-installed package and start writing code

Choose from any of Pulumi’s cloud provider packages in the Registry
Install into your project with go get [package-url]
Reference a namespace in the newly-installed package and start writing code

Notably, you don’t need to know how a Pulumi Package was authored to use it. Using any Pulumi Package is the same regardless of whether that package contains native cloud resources, custom components, resources that came from a bridged Terraform provider, or anything of the other many possibilities. No matter what, the same idiomatic Pulumi IaC programming model is used to interact with those resource types and objects.

Creating Pulumi Packages

You can use Pulumi successfully without ever creating a Pulumi Package. But, we’d love to have you join the community of Pulumi Package authors! Here’s a quick overview of how Pulumi Packages work and how to create one.

A Pulumi Package is comprised of two inputs: a Pulumi schema, which describes the resource model the package exposes, and the Resources or Components described in that schema. From those inputs, Pulumi Packages can generate language-specific SDKs and documentation and then package those into SDK packages (npm, NuGet, and Python packages) that can be published to a public or private package registry. Along the way, a resource provider plugin is also created. The plugin is used by the Pulumi engine to instantiate, configure, and destroy the Resources or Components defined by the package.

There are several different ways to author a Pulumi Package depending on the kind of cloud infrastructure you want to define:

Native Pulumi Provider Package: Use the full features of the Pulumi resource model to create a provider for a new cloud platform. Examples: the kubernetes, azure-native, and google-native packages.
Bridged Provider Package: Take an existing resource provider from another supported ecosystem (like a Terraform provider), and bridge it to be exposed as a Pulumi Package. Examples: the aws, tls, and cloudflare packages.
Component Package: Write a Pulumi Component in your language of choice and expose it to users in all Pulumi languages. Example: the eks package.

Together, these options provide a combination of breadth and depth for the kinds of resources and components that can be exposed to Pulumi developers via Pulumi Packages. In the weeks ahead, we have many improvements underway to make the authoring and publishing of packages much easier. We also look forward to your feedback on the overall experience!

👉 Author your first Pulumi Package

Keep exploring

Pulumi Packages are just one part of Pulumi’s Cloud Engineering Platform. Watch the PulumiUP! event for more.

👉 PulumiUP!

Build your perfect interface for the cloud: Automation API

Alex Mullans — Tue, 20 Apr 2021 00:00:02 -0700

Pulumi Automation API exposes the full power of infrastructure as code through a programmatic interface, instead of through CLI commands. With Automation API, you can create software that has the capability to provision and configure infrastructure: create, update, configure, and destroy infrastructure dynamically. You can create new classes of Software-as-a-Service that provision unique infrastructure for every customer. You can build the perfect cloud interface for your organization, combining all of your infrastructure knowledge and best practices into a purpose-built tool, whether it’s a SaaS offering, an internal web app, a purpose-built CLI tool, a CI/CD integration, or something else entirely.

We announced the general availability of Automation API today at PulumiUP and we’re excited to see what you create with it!

Making self-service infrastructure platforms easier

So, why might you want to create your own interface to the cloud? Let’s take a step back and review the journey that many organizations take when they embrace the modern cloud. Infrastructure engineers across the organization will typically start designing and iterating on infrastructure, using Pulumi interactively on the command line or sometimes using a graphical interface like the AWS Console and Azure Portal or an imperative CLI like aws or az. In this early phase, rapid iteration is important, as infrastructure engineers try different combinations of services, different configurations, and more until they land on a configuration that they want to make ready for production.

When they’re ready to go to production, the easy exploration and rapid iteration afforded by the cloud provider’s graphical and CLI interfaces become less important. Instead, new concerns about scalability, security, compliance, and cost control come to the forefront. At this point, it’s common to turn to infrastructure as code templates. Pulumi’s modern IaC allows infrastructure engineers to codify the infrastructure they’ve designed into a code file that can be managed like any other piece of software: checked into source control, forked and modified, and executed when needed by an engineer or application developer using the pulumi CLI in a terminal. In some organizations, this is the end of the journey: application developers start from the canonical infrastructure as code solutions created by their infrastructure team and modify them as needed to suit their unique needs. In those organizations, Pulumi’s policy-as-code offering can help infrastructure teams keep application teams “within the guardrails” as they customize their infrastructure.

However, not all organizations want to expose the full complexity of infrastructure, or an infrastructure as code tool like Pulumi, to their application developers. This is where Pulumi’s Automation API and the custom cloud interfaces that can be created with it come into play. Automation API allows infrastructure teams to provide self-service tools (e.g. developer portals, purpose-built CLIs, custom frameworks, and CI/CD workflows) for building, deploying, and managing infrastructure that offer the right levels of complexity and customization for the tool’s target audience. With Automation API, infrastructure teams scale their knowledge and expertise across the entire organization by encoding it into the tool they create. Application teams get self-service access to infrastructure without having to understand every underlying resource and option. Security and compliance teams get peace of mind that all of the organization’s infrastructure is created and managed through an approved tool that follows the right processes and procedures.

Unlocking new classes of Software-as-a-Service offerings

Before Automation API, most Software-as-a-Service (SaaS) products used multi-tenant infrastructure: infrastructure that’s shared across many of the SaaS’s customers. They did so because it wasn’t possible to create and manage unique infrastructure for each customer. But, some SaaS customers—especially in regulated industries like healthcare and finance—were reluctant to use multi-tenant SaaS offerings because of security and performance concerns, like data leakage between tenants or inadequate performance when a neighboring tenant was heavily using the system.

Pulumi’s Automation API unlocks new customers for existing SaaS products and entirely new classes of single-tenant SaaS products, like Pulumi customers CockroachDB and Snowflake. For these leading providers of cloud software, Automation API unlocks the capability to make infrastructure a fundamental part of their offering. With Automation API, they can create and manage unique infrastructure for every customer, even as they scale to 1,000s of customers. They can provide solutions that weren’t feasible with multi-tenant infrastructure or bring existing solutions to customers with higher needs for security, performance, and compliance. In short, they can leverage cloud infrastructure as a competitive advantage.

Start using Automation API: from `pulumi up` to `stack.up()`

To get started with Automation API, you can take any existing Pulumi program. Instead of running commands like pulumi up interactively in a terminal to provision your infrastructure, with Automation API you write an application that calls functions like stack.up(). Then, when you deploy that application and give your customers access to it, they can stand up infrastructure when they’re ready.

At its core, the Automation API enables you to use the entire Pulumi engine as an SDK for the applications you create. Like all of Pulumi, Automation API is available in multiple languages, so you can create applications that use it in TypeScript/JavaScript, Python, Go, and C#.

👉 Get started with the Automation API User Guide

Automation API in the wild

In the short while that Automation API has been in preview, we’ve already seen it used for a wide variety of custom platforms and other automation-centric use cases like CI/CD. Automation API powers custom infrastructure platforms—like Mercedes-Benz Research & Development’s platform—that empower teams across the organization to stand up the infrastructure they need in the cloud when they need it. It’s also used to operate Pulumi natively in automation and CI/CD environments; if you’ve used Pulumi’s GitHub Action, you’ve already used Automation API. Read on to see how Automation API has been used across languages, scenarios, and ecosystems.

Pulumipus’ Self-Service Platform, a web app in Python

Today at PulumiUP, Komal Ali—one of the primary authors of Automation API—will show “Pulumipus’ Self-Service Platyform”: a Python-based interface for creating customized static sites, databases, virtual machines, and more. The demo shows how easy it is to build a self-service web interface for infrastructure by combining Pulumi infrastructure as code and a Flask web app. The code is open source and available on GitHub.

Ploy, a CLI in Go

It’s also possible to create custom command-line interfaces with Automation API. Lee Briggs, a Pulumi community engineer, shows one such interface in this video, called Ploy. Ploy is a Heroku-like interface for deploying Docker images to a Kubernetes cluster. Ploy is fully open source on GitHub and combines Pulumi infrastructure with a Go CLI using Promptui.

Python IaC in your Jupyter notebook

Many data scientists work in Jupyter notebooks, where they can interactively execute Python code as they train models, create analyses, and more. Because these notebooks use Python, it’s possible to use Pulumi’s Python SDK and Automation API to create infrastructure interactively. Pulumian Sophia Parafina showed how to do so in this blog post from earlier this year. Examples of using Pulumi in a Jupyter notebook are available in our open source examples repository on GitHub.

Pulumi Resource Type for Concourse, a CI/CD extension in Go

Some users’ perfect interface to the cloud is a continuous integration and delivery (CI/CD) platform that builds applications and infrastructure together in a unified pipeline. Pulumi’s Automation API is a key component that has helped many of our customers plug Pulumi into their CI/CD platform of choice. One such extension is the Pulumi Resource Type for Concourse, which allows users of the Concourse CI product to add infrastructure to their pipelines. Authored by Pulumi community member Ringo De Smet, the extension is open source on GitHub.

Pulumi GitHub Action, a CI/CD extension in TypeScript

Pulumi works great with GitHub Actions, one of the most popular CI/CD systems, thanks to our GitHub Action, which Automation API also powers. Co-developed by Pulumi community member Simen A. W. Olsen and Pulumian Paul Stack, Pulumi’s GitHub Action gives you complete control over Pulumi in your CI/CD and other automation workflows. Read more about how it was created using Automation API or see the TypeScript source code on GitHub.

👉 See more examples of Automation API in action

Keep exploring

Automation API is just one part of the Pulumi Cloud Engineering Platform, announced today at PulumiUP. Watch the PulumiUP event for more.

👉 Get started with the Automation API User Guide 👉 PulumiUP

Unify app and infra delivery with Pulumi's CI/CD Assistant

Alex Mullans — Tue, 20 Apr 2021 00:00:01 -0700

Pulumi’s CI/CD Assistant helps you bring your infrastructure under version control and create a continuous integration and delivery pipeline that deploys changes to your cloud applications and infrastructure whenever you make a change in source control. Using CI/CD secures your production delivery while ensuring that every deployment is expressed in a committed Pulumi program and quickly reflected in your deployed infrastructure. With the CI/CD Assistant, it’s easier than ever before to set up version control and a CI/CD pipeline for your favorite CI/CD system, even if you’re new to CI/CD workflows.

Since its launch late last year, the CI/CD Assistant has been available to organizations using the Team and Enterprise offerings; today at PulumiUP, we announced that it’s now also available to all teams, including Team organizations. Get started with the CI/CD Assistant on any stack’s settings page, or learn more in the docs.

Get your cloud applications and infrastructure to production faster and safer

Building source code safely in version control and deploying using continuous integration and delivery (CI/CD) pipelines are best practices of cloud engineering. Using version control creates a full history of changes to Pulumi infrastructure as code programs you write: when the change was made, who made it, and what was changed. Version control systems also provide an easy mechanism to revert to an old infrastructure configuration: you can restore a past version of your infrastructure as code and redeploy it, effectively rolling back by rolling forward. Adding a CI/CD pipeline adds even more benefit by accelerating the deployment of changes. The CI/CD pipeline for your cloud code can deploy updates on every change, on a schedule, or in many CI/CD systems by an external event or workflow you’ve configured. A well-configured CI/CD pipeline is also much more secure: you can focus on restricting access to production deployments, generating temporary access keys for the CI/CD environment, and also also use the Pulumi Secrets feature to safeguard and minimize access to any and all secrets you use to deploy.

The CI/CD Assistant helps you realize all these benefits by getting version control and CI/CD configured for your infrastructure stacks. The Assistant walks you through a 4-step guided procedure:

Configuring version control, so Pulumi can show you the code history associated with each update
Configuring your CI/CD system to access Pulumi, including any secrets Pulumi safeguards for your stack
Creating a CI/CD workflow file that shows how to run Pulumi in your CI/CD system
Validating that everything was configured correctly

When you’ve configured version control and CI/CD on a Pulumi stack, you get additional helpful metadata in the Pulumi Service: each update shows the commit, branch, pull request (if applicable), and build number - all with links to your version control and CI/CD systems so you can dig into additional detail.

👉 Learn more

Support for many CI/CD providers

The CI/CD Assistant delivers on the Pulumi Cloud Engineering Platform’s promise of your cloud, your language, your way, by providing broad support for over a dozen of the most popular CI/CD services:

If you use a version control or CI/CD provider that’s not on this list, you can file a feature request. We’d love to hear from you!

👉 Request a new version control or CI/CD provider

Keep exploring

This announcement is just one part of the story. Watch the PulumiUP event for more.

👉 PulumiUP

Easily bring your team to Pulumi with SAML SSO and SCIM

Alex Mullans — Tue, 20 Apr 2021 00:00:00 -0700

Pulumi’s Cloud Engineering Platform helps teams of all sizes deliver and manage cloud apps and infrastructure. In the Pulumi Service, everyone on the team can see the infrastructure the team is responsible for, when it was last deployed, how it’s configured, and more. You can see a full breakdown of the infrastructure and understand how the team brings together individual cloud services to create applications. When you bring your teams together on Pulumi, you get a “single pane of glass” over all the cloud applications and infrastructure managed with Pulumi.

Most teams larger than a few people define their team members, and the groups they’re a part of, using an Identity Provider (IdP) like Okta, Microsoft’s Azure Active Directory, or Google Cloud Identity & Access Management. Defining groups allows you to set policies based on logical groupings of teammates, such as giving different capabilities to individual developers versus infrastructure administrators. Pulumi works seamlessly with these IdPs (and many more) by providing Single Sign-On with SAML and user and group synchronization via SCIM 2.0. When you use both SAML SSO and SCIM, you can manage the users who should have access to Pulumi and the Pulumi teams they belong to entirely from your IdP. Managing users and teams this way helps keep your organization more secure by ensuring a single point of control over the users who have access to manage and update your infrastructure. It also saves you time by centralizing all of your identity and access management workflows in your identity provider.

SAML SSO and SCIM support are only available in Pulumi Enterprise. See pricing for more details.

SAML SSO

In a world of many cloud applications and service providers, IT departments can use single sign-on with SAML to make it easier for their employees to sign in and be productive. In addition, if an employee leaves, all of that employee’s access can be disabled from a single pane of glass, rather than having to manually update each application and service provider separately. Pulumi’s support for single sign-on with SAML provides this functionality. We support any IdP that implements the SAML 2.0 specification, with specific integrations for Azure Active Directory, Okta, and GSuite.

To configure single sign-on with SAML for your Pulumi organization, use the guide that matches your IdP:

After configuring your IdP, any and all changes, either to your cloud applications and infrastructure, or your account’s settings, will be tracked and associated with the user who performed that activity. The end result is full end-to-end visibility and attestation for important changes to your Pulumi organization.

User and group synchronization via SCIM

After you’ve configured single sign-on with SAML, you can take it one step further by synchronizing your users and groups using SCIM. Synchronization extends the power of single sign-on by enabling organization administrators to use their IdP to manage the users who have access to Pulumi and which teams each user belongs to. Granting and revoking user access is seamless and automatically provides the right level of permissions to critical infrastructure.

To configure user and group synchronization via SCIM, use the guide that matches your IdP:

Distributed, least privilege access permissions model

In addition to support for standards like SAML and SCIM, the Role-Based Access Control (RBAC) permissions model enables you to set up a least privilege access model for your infrastructure resources. In such a model, all access to infrastructure is specifically granted to individual users by creating Teams that have the appropriate level of access (Read, Write, or Admin) to a set of Stacks, then adding users to those teams. Team membership can be managed manually by Team Admins or automatically using SCIM group synchronization.

Learn how to create Teams in the Teams docs.

Next steps

The Pulumi Cloud Engineering Platform’s support for the SAML and SCIM standards for SSO, user synchronization, and group synchronization makes it easy to ensure every team member has the right access and the right permissions, allowing you to empower the right teammates while still applying the principle of least privilege. Here’s how to get started:

If you’re an existing Pulumi customer, you can enable SAML in your organization settings.
If you’re new to Pulumi, start your trial now or contact sales to request a demo.

Easily bring your team to Pulumi with SAML SSO and SCIM

Alex Mullans — Wed, 24 Feb 2021 00:00:00 +0000

The Pulumi Service helps teams of all sizes deliver and manage cloud apps and infrastructure. In the console, everyone on the team can see the infrastructure the team is responsible for, when it was last deployed, how it’s configured, and more. They can see a full breakdown of the infrastructure as well, so they can understand how the team brings together individual cloud services to create their applications. When you bring your teams together on the Pulumi Service, you can provide a “single pane of glass” over all the infrastructure that you manage with Pulumi.

Most teams larger than a few people define their team members, and the groups they’re a part of, using an Identity Provider (IdP) like Okta, Microsoft’s Azure Active Directory, or Google Cloud Identity & Access Management. The Pulumi Service works seamlessly with these IdPs (and many more) by providing Single Sign-On with SAML and user and group synchronization via SCIM 2.0.

When you use both SAML SSO and SCIM, you can manage the users who should have access to Pulumi and the Pulumi teams they belong to entirely from your IdP. Managing users and teams this way helps keep your organization more secure by ensuring a single point of control over the users who have access to manage and update your infrastructure. It also saves you time by centralizing all of your identity and access management workflows in your identity provider.

SAML SSO

In a world of many cloud applications and service providers, IT departments can use single sign-on with SAML to make it easier for their employees to sign in and be productive. In addition, if an employee leaves, all of that employee’s access can be disabled from a single pane of glass, rather than having to manually update each application and service provider separately. The Pulumi Service’s support for single sign-on with SAML provides this functionality. We support most IdPs that implement the SAML 2.0 specification.

To configure single sign-on with SAML for your Pulumi organization, use the guide that matches your IdP:

User and group synchronization via SCIM

After you’ve configured single sign-on with SAML, you can take it one step further by synchronizing your users and groups using SCIM. Synchronization extends the power of single sign-on by enabling organization administrators to use their IdP to manage the users who have access to the Pulumi Service and which Pulumi teams each user is a member of. Granting and revoking user seat licenses is seamless and easily provides the right level of permissions to critical infrastructure.

To configure user and group synchronization via SCIM, use the guide that matches your IdP:

Next steps

The Pulumi Cloud’s support for the SAML and SCIM standards for SSO, user synchronization, and group synchronization make it easy to ensure every team member has the right access and the right permissions. Here’s how to get started:

If you’re an existing Pulumi customer, you can enable SAML in your organization settings.
If you’re new to Pulumi, start your trial now or contact sales to request a demo.

Keep your secrets secure, by default

Alex Mullans — Fri, 19 Feb 2021 00:00:00 +0000

An unauthorized user gaining access to your infrastructure can be catastrophic: data can be stolen or leaked, security holes can be exploited, and more. That risk makes it critical to keep the infrastructure secrets—the passwords, access tokens, keys, and so on—well-protected. This is particularly true in automated systems, like continuous integration and delivery and infrastructure-as-code systems.

Pulumi’s built-in secret storage

Pulumi takes these risks seriously. That’s why we built the Pulumi platform to be secure by default. It protects all secret data using Pulumi’s built-in secret storage. This secret store “just works”, regardless of whether you use our hosted Pulumi Service, the Self-Hosted Pulumi Service, or the self-managed backend. The Pulumi Service protects secrets automatically when you create a stack with pulumi new or pulumi stack init. If you use a self-managed backend like AWS S3 or Google Cloud Storage, secrets are protected by a passphrase that you choose when you start a new stack with pulumi new. In all cases, every Pulumi stack gets a unique encryption key.

What secrets are protected

Pulumi protects all of the state metadata that it knows about your infrastructure with encryption: encryption in transit when it’s sent to your backend and encryption at rest (if you’re using the hosted or self-hosted Pulumi Service). This base layer of protection applies to all state, including both non-secret and secret metadata. Secret storage is the next level of protection. It ensures that secret values are never exposed in plaintext in any file that Pulumi writes to your disk. Secrets are identified in a few ways:

All Pulumi providers—both native providers like azure-native and Terraform-based providers like Datadog—know which of their outputs should be secret and mark them as such. This ensures that you’re secure by default when you start using a new provider. There’s no need to pick which outputs are secret or worry that you’ve missed one.
You can mark other Pulumi outputs as secret.
In addition to Pulumi inputs and outputs, you can also mark Pulumi configuration values as secret.

Any secret identified by you or by Pulumi is fully protected in the secret storage and not stored in plaintext on disk. This prevents leaks of secrets when files are committed to a source code repository or uploaded to a continuous integration and delivery system.

Integration with other secret stores

For smaller organizations, the secret storage built into Pulumi is usually sufficient. In larger organizations however, it’s likely that a centralized secret management system is already in use, like AWS Key Management System, Azure Key Vault, Google Cloud Key Management, or Hashicorp Vault. Pulumi integrates with each of these systems to make it easy to protect secrets in the way required by the organization.

Next steps

It’s easy to get started with secrets management in Pulumi, because there’s nothing extra you need to do. Begin with any one of our get started tutorials and Pulumi will automatically protect your secrets along the way.

Learn more about Pulumi’s secret storage in our docs

Pulumi Blog: Alex Mullans

Dec. 8 releases: K8s examples, exclude protected resources from destroy, easier invites to the Pulumi Service

Pulumi Registry, Pulumi Packages, & integrations

Helm Release resource supports local charts

New examples for the Kubernetes native provider

New resources in the AWS Native provider

New resources in the Azure Native provider

Support for Kubernetes 1.23

Pulumi CLI and core technologies

Exclude protected resources from pulumi destroy

Pulumi Package plugins can now be hosted on GitHub Releases

Pulumi Service & Pulumi.com

More ways to share invites to your organization

Self-Hosted Pulumi Service can disable email sign-up and login

Nov. 17 releases: `dependOn` Helm charts, new Elastic Cloud provider, functions support outputs, set the CLI's default organization

Pulumi Registry, Pulumi Packages, & integrations

dependsOn can now await a fully-deployed Helm Chart resource

New provider: Elastic Cloud

AWS Native provider supports auto-naming

New resources in the AWS Native provider

New resources in the Azure Native provider

Pulumi CLI and core technologies

Functions now support outputs

--json flag for pulumi {update|destroy|preview|refresh}

Python 3.6 support will be deprecated soon

Pulumi Service & Pulumi.com

Set the default organization for newly-created stacks

Oct. 27 releases: Pulumi Registry, K8s Operator 1.0

Cloud Engineering Summit

Pulumi Registry, Pulumi Packages, & integrations

Pulumi Registry: your window to the cloud

New Pulumi Components for AWS, Azure, and popular Helm charts

Pulumi Kubernetes Operator 1.0

New resources in the AWS Native provider

New resources in the Azure Native provider

Pulumi CLI and core technologies

Resource methods for Pulumi Packages

Automation API: delete, unprotect state in .NET

Pulumi Service & Pulumi.com

Automated audit log export

Introducing Pulumi Registry: Deploy Cloud Infra Faster

Find & Deploy Cloud Infrastructure with Pulumi Registry

Pulumi Providers: The Most Complete Cloud IaC Solution

Use Any Terraform Provider with Pulumi Registry

Create & Publish Custom Pulumi Providers

Pulumi Components: Prebuilt IaC for Faster Cloud Deployments

Deploy a Production-Ready EKS Cluster with Pulumi (in 15 Lines)

Build a Serverless App with AWS Lambda & API Gateway in 21 Lines

Create & Publish Your Own Pulumi Components

What’s Next for Pulumi Registry?

Oct. 6 releases: AWS Native Provider, more refresh options

New and updated cloud providers

AWS Native Provider, powered by the AWS Cloud Control API

Using AWS Graviton2 processors in AWS Lambda functions

New resources in the Azure Native provider

Pulumi CLI and core technologies

New project configuration option to refresh before all write operations

Output property enhancements

Sep. 16 releases: Helm Release, pulumi about, easier invites

New and updated cloud providers

New Helm Release resource for Kubernetes

New providers: MinIO and Snowflake

New resources in the Azure Native provider

Pulumi CLI and core technologies

New pulumi about command for easier troubleshooting

pulumi config set hides secret values

Dynamic providers in Python can now given a custom name

Automation API in .NET can use plugin installation options exact and server

Schema checker for Pulumi Packages

Pulumi Service and Pulumi.com

Organization invites now send you to the correct identity provider

August 18 releases: EKS, Pulumi Component methods, dependsOn

New and updated cloud providers

EKS Pulumi Package supports more options for managed node groups

New resources in the Azure Native provider

Pulumi CLI and core technologies

Methods for Pulumi Component Packages

dependsOn now works for Pulumi Component Packages, and more

Aliases can now be used with providers

Pulumi Service and Pulumi.com

Helm `Release` resource supports local charts

Exclude protected resources from `pulumi destroy`

`dependsOn` can now await a fully-deployed Helm `Chart` resource

`--json` flag for `pulumi {update|destroy|preview|refresh}`

New `pulumi about` command for easier troubleshooting

`pulumi config set` hides secret values

Automation API in .NET can use plugin installation options `exact` and `server`

`dependsOn` now works for Pulumi Component Packages, and more

Kubernetes: new `skipAwait` option

New `replaceOnChanges` resource option

`pulumi watch` now supports a `--path` option

Start using Automation API: from `pulumi up` to `stack.up()`