Pulumi Blog: Matt Jeffryes

Announcing the 1.0 release of AWS CDK on Pulumi

Matt Jeffryes — Mon, 02 Dec 2024 08:00:00 -0700

At Pulumi, we’re committed to delivering the widest range of cloud infrastructure building blocks for use in your cloud engineering projects. In 2022, we introduced preview support for integrating AWS CDK constructs into Pulumi programs and today we’re happy to announce the 1.0 release of our pulumi-cdk library for typescript. This first stable version completes support for common CDK features enabling you to deploy almost any CDK construct with Pulumi.

The pulumi-cdk library provides access to the many high-level libraries (‘constructs’) built by service teams at AWS and by the AWS CDK community. However, there were some limitations in the preview version of the library that prevented using some constructs and complicating migration of existing CDK stacks. Uncertainty about which constructs were supported has limited adoption of this rich ecosystem by Pulumi users.

With the new 1.0 release, pulumi-cdk has greatly expanded compatibility with all CDK features. Users can now confidently use Pulumi to deploy constructs they have written themselves or any of the 1600+ constructs in AWS’s construct hub. We’re excited to bring Pulumi users access to all the high level interfaces curated by AWS and the CDK community.

Additionally, existing AWS CDK users now have an easier path to transition from CloudFormation to Pulumi, unlocking faster deployments, integration with Pulumi’s vast provider ecosystem and all of the features of Pulumi’s Cloud Engineering Platform (like Policy as Code, Audit Logs, Drift detection, Secrets, and much more).

Deploying CDK Constructs with Pulumi

To deploy existing AWS CDK Constructs using Pulumi:

Create a class that derives from pulumicdk.Stack (which itself is derived from awscdk.Stack).
In the constructor, use any AWS CDK constructs from existing libraries such as aws-cdk-lib
Create a pulumicdk.App instance that constructs an instance of your stack.

For example, the following program deploys an AWS Apprunner CDK Construct using Pulumi:

import * as pulumi from '@pulumi/pulumi';
import * as pulumicdk from '@pulumi/cdk';
import { Service, Source } from '@aws-cdk/aws-apprunner-alpha';

class AppRunnerStack extends pulumicdk.Stack {
 url: pulumi.Output<string>;

 constructor(id: string, options?: pulumicdk.StackOptions) {
 super(id, options);

 const service = new Service(this, 'service', {
 source: Source.fromEcrPublic({
 imageConfiguration: { port: 8000 },
 imageIdentifier: 'public.ecr.aws/aws-containers/hello-app-runner:latest',
 }),
 });

 this.url = this.asOutput(service.serviceUrl);
 }
}

const app = new pulumicdk.App('app', (scope: pulumicdk.App): pulumicdk.AppOutputs => {
 const stack = new AppRunnerStack('teststack');
 return {
 url: stack.url,
 };
});
export const url = app.outputs['url'];

Deploy with a pulumi up:

> pulumi up

Updating (dev)

View Live: https://app.pulumi.com/lukehoban/pulumi-cdk-apprunner/dev/updates/1

 Type Name Status
 + pulumi:pulumi:Stack pulumi-cdk-apprunner-dev created
 + └─ cdk:index:StackComponent teststack created
 + └─ cdk:construct:Service teststack/adapter/service created
 + └─ aws-native:apprunner:Service service6D174F83 created

Outputs:
 url: "2ez3iazupm.us-west-2.awsapprunner.com"

Resources:
 + 4 created

And curl the endpoint:

> curl https://$(pulumi stack output url)

 ______ __ __ __ _ __
 / ____/___ ____ ____ __________ _/ /___ __/ /___ _/ /_(_)___ ____ _____/ /
 / / / __ \/ __ \/ __ `/ ___/ __ `/ __/ / / / / __ `/ __/ / __ \/ __ \/ ___/ /
/ /___/ /_/ / / / / /_/ / / / /_/ / /_/ /_/ / / /_/ / /_/ / /_/ / / / (__ )_/
\____/\____/_/ /_/\__, /_/ \__,_/\__/\__,_/_/\__,_/\__/_/\____/_/ /_/____(_)
 /____/

 Congratulations, your service has successfully deployed on AWS App Runner.

Open it in your browser at https://2ez3iazupm.us-west-2.awsapprunner.com/

Try the workshop at https://apprunnerworkshop.com
Read the docs at https://docs.aws.amazon.com/apprunner

Pulumi takes CDK apps to the next level

Combine CDK constructs with any of the features of Pulumi programs to deploy faster, easier and to any cloud. Use Pulumi functions and stack references to connect to pre-existing infrastructure and mix in resources from any provider to bring all of your infrastructure under management.

For example, we can use CDK’s ecs_patterns to quickly create a loadbalanced Fargate service, but route traffic to it with a record in an existing CloudFlare DNS zone.

import * as path from 'path';
import * as acm from 'aws-cdk-lib/aws-certificatemanager';
import * as aws from '@pulumi/aws';
import * as pulumi from '@pulumi/pulumi';
import * as pulumicdk from '@pulumi/cdk';
import * as cloudflare from '@pulumi/cloudflare';
import * as ecs from 'aws-cdk-lib/aws-ecs';
import * as ecsPatterns from 'aws-cdk-lib/aws-ecs-patterns';

let config = new pulumi.Config();
let accountId = config.requireSecret("cloudflare_account_id");
let zoneName = config.require("cloudflare_zone_name");
let certificateId = config.require("certificate_id");
let certificateKey = config.requireSecret("certificate_key");

class CloudFlareStack extends pulumicdk.Stack {
 constructor(app: pulumicdk.App, id: string) {
 super(app, id);

 // First Load the zone and certificate from CloudFlare
 ///////////////////////////////////////////////////////

 // Read the CloudFlare zone for the domain
 const zone = cloudflare.getZoneOutput({
 accountId,
 name: zoneName,
 });

 // read the certificate from CloudFlare
 const caCertificate = cloudflare.getOriginCaCertificateOutput({
 id: certificateId,
 });

 // Import the certificate in ACM
 const cert = new aws.acm.Certificate('import', {
 privateKey: certificateKey,
 certificateBody: caCertificate.certificate,
 });

 // Create a L2 reference from the cert arn
 const acmCert = acm.Certificate.fromCertificateArn(this, 'cert', pulumicdk.asString(cert.arn));

 // Next, use ECS Patterns to create a loadbalanced fargate service with the correct certificate
 ///////////////////////////////////////////////////////////////////////////////////////////////

 const loadBalancedFargateService = new ecsPatterns.ApplicationLoadBalancedFargateService(this, 'Service', {
 certificate: acmCert,
 redirectHTTP: true,
 taskImageOptions: {
 image: ecs.ContainerImage.fromRegistry("amazon/amazon-ecs-sample"),
 },
 });

 const alb = loadBalancedFargateService.loadBalancer;

 // Finally, Create a record in CloudFlare that directs to the service's loadbalancer
 ////////////////////////////////////////////////////////////////////////////////////

 new cloudflare.Record('alb', {
 name: 'cdk-alb',
 type: 'CNAME',
 zoneId: zone.zoneId,
 content: this.asOutput(alb.loadBalancerDnsName),
 proxied: true,
 });
 }
}

class MyApp extends pulumicdk.App {
 constructor() {
 super('app', (scope: pulumicdk.App) => {
 new CloudFlareStack(scope, 'cloudflare');
 });
 }
}

new MyApp();

Pulumi let’s you read and create any resource type across thousands of different cloud service providers and integrate them with your CDK stacks. Building CDK apps on Pulumi brings a host of other benefits too, including:

Customize resource options (eg. to protect a database against accidental deletion or deploy to multiple regions in the same program)
Detect and manage drift with Pulumi deployments
Track and store user actions and change history with Audit Logs
And a host of other benefits from Pulumi Cloud

Expanded Features in pulumi-cdk 1.0

The 1.0 version of pulumi-cdk adds support for key CDK features such as: assets, custom resources, nested stacks, and aspects, so most existing CDK constructs will work with Pulumi without any modification. We’ve also eliminated the need to call cdk bootstrap before running pulumi up by integrating a custom Pulumi Synthesizer.

Using CDK Aspects

Pulumi CDK now supports CDK aspects. For example, you can add the cdk-nag aspect to your stack to enforce best practices:

import * as pulumicdk from '@pulumi/cdk';
import * as s3 from 'aws-cdk-lib/aws-s3';
import { AwsSolutionsChecks } from 'cdk-nag';

const app = new pulumicdk.App('app', (scope: pulumicdk.App): pulumicdk.AppOutputs => {
 const stack = new pulumicdk.Stack('example-stack');
 Aspects.of(stack).add(new AwsSolutionsChecks({ verbose: true }));

 new s3.Bucket(this, 'bucket');
});

Example Output

[Error at /test-stack/bucket/Resource] AwsSolutions-S1: The S3 Bucket has server access logs disabled. The bucket should have server access logging enabled to provide detailed records for the requests that are made to the bucket.
[Error at /test-stack/bucket/Resource] AwsSolutions-S10: The S3 Bucket or bucket policy does not require requests to use SSL. You can use HTTPS (TLS) to help prevent potential attackers from eavesdropping on or manipulating network traffic using person-in-the-middle or similar attacks. You should allow only encrypted connections over HTTPS (TLS) using the aws:SecureTransport condition on Amazon S3 bucket policies.

CDK assets

CDK’s assets make it easy to bundle code and other files your application needs into an S3 bucket or a Docker container, eliminating extra steps from your build pipeline. For example you can use Docker container asset to deploy a Fargate service from local directory. CDK will build the container image and push to ECR on pulumi up.

import * as pulumicdk from '@pulumi/cdk';
import * as aws from '@pulumi/aws';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as ecs from 'aws-cdk-lib/aws-ecs';
import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns';

const app = new pulumicdk.App('app', (scope: pulumicdk.App) => {
 const stack = new pulumicdk.Stack('example-stack');

 const vpc = new ec2.Vpc(stack, 'MyVpc');
 const cluster = new ecs.Cluster(stack, 'fargate-service-autoscaling', { vpc });

 // Create Fargate Service
 const fargateService = new ecs_patterns.NetworkLoadBalancedFargateService(this, 'sample-app', {
 cluster,
 taskImageOptions: {
 image: ecs.ContainerImage.fromAsset(path.join(__dirname, './'), {
 // assetName is required and is used in the name of the ecr repository that is created
 assetName: 'cdk-fargate-example',
 }),
 },
 });
});

Future direction

Currently AWS CDK on Pulumi is supported only for TypeScript users, but we’re eager to bring the benefits of the CDK ecosystem to all Pulumi languages in the future. You can comment and upvote on this tracking issue, to let us know this is important to you.

We’re also exploring options for importing state from CDK applications that are already deployed with CloudFormation to enable seamless migration for existing CDK users. Please let us know on this tracking issue if you would be interested in testing this with us.

Summary

The 1.0 release of AWS CDK support for Pulumi marks a significant expansion of opportunities to interoperate between the AWS CDK and Pulumi ecosystems. Existing AWS CDK users can now more easily adopt the full power, speed and flexibility of the Pulumi Cloud Platform and its diverse ecosystem. And existing Pulumi users can benefit from the many expert curated constructs and libraries in the CDK community. We’re excited to see what you’ll build with these two great cloud technologies together.

The AWS CDK on Pulumi project is open source at https://github.com/pulumi/pulumi-cdk and on NPM at https://www.npmjs.com/package/@pulumi/cdk. Get started today!

Pulumi AWS Cloud Control Provider is Generally Available

Tejitha Raju — Tue, 01 Oct 2024 09:00:00 -0700

We’re excited to announce the general availability of the new AWS Cloud Control Provider for Pulumi (previously “AWS Native”). AWS is one of the most-used cloud providers across the Pulumi ecosystem, and we are committed to providing the fastest possible access to new AWS capabilities to all Pulumi users. This provider delivers on that promise by providing native support for all resources in the AWS Cloud Control APIs. As part of this launch, we are also renaming the provider from “AWS Native” to “AWS Cloud Control Provider,” based on customer feedback during the preview phase, to provide users with a broader range of advanced cloud management tools exposed by the AWS Cloud Control API. This provider includes the following features and benefits,

Day 1 support for AWS Resources: Provides full coverage of the AWS Cloud Control API, typically on the day of the launch, ensuring users can adopt new features immediately.
Complementary Pulumi Experience: Works alongside Pulumi’s AWS Provider, enhancing Infrastructure as Code projects with the latest AWS capabilities without requiring significant changes to existing setups.
Extended support for third-party resources: Our platform offers enhanced flexibility and interoperability by supporting third-party resources available in the CloudFormation Registry, such as Atlassian, MongoDB, Datadog, and more.
Easy CloudFormation to Pulumi Migration: cf2pulumi provides the ability to migrate existing CloudFormation templates into Pulumi programs in your favorite language.

At Pulumi, we’re dedicated to empowering our customers with the tools they need to innovate and thrive in the cloud. The launch of the AWS Cloud Control Provider represents our unwavering commitment to providing best-in-class solutions that simplify cloud management, streamline operations, and drive business success. The Pulumi AWS Cloud Control Provider can be used side-by-side with the standard Pulumi AWS provider and nearly 200 additional Pulumi resource providers that cover a wide variety of cloud and SaaS platforms.

Using Pulumi AWS Cloud Control Provider with the Pulumi AWS Provider

Let’s walk through an example of using the AWS Cloud Control Provider alongside the Pulumi’s standard AWS Provider. This example shows how to create an S3 bucket with the standard AWS provider and deploy a Lambda function using the AWS Cloud Control provider. The Lambda function is set up to use the S3 bucket’s name and process events accordingly. This example showcases how both providers can work together, leveraging the standard AWS Provider for widely supported resources, while using AWS Cloud Control for the latest features and APIs provided by AWS.

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awscc from "@pulumi/aws-native";

// ARN of the IAM role that the Lambda function will assume
const roleArn = "arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test";

// Create an S3 bucket using the AWS provider
const bucket = new aws.s3.Bucket("myBucket");

// Create the AWS Lambda function using the AWS Cloud Control provider
const lambdaFunction = new awscc.lambda.Function("myLambdaFunction", {
 functionName: "my-lambda-function",
 role: "arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test",
 runtime: "nodejs18.x",
 handler: "index.handler",
 code: {
 zipFile: `
 exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
 };
 `,
 },
 environment: {
 variables: {
 BUCKET_NAME: bucket.bucket,
 },
 },
});

// Export the Lambda function ARN and the S3 bucket name
export const lambdaFunctionArn = lambdaFunction.arn;
export const s3BucketName = bucket.bucket;

import pulumi
import pulumi_aws as aws
import pulumi_aws_native as aws_cloudcontrol
# Create an S3 bucket using the standard AWS provider
my_bucket = aws.s3.Bucket("myBucket")

# Create the Lambda function using the AWS Cloud Control provider
my_lambda_function = aws_cloudcontrol.lambda_.Function("myLambdaFunction",
 function_name="my-lambda-function",
 role="arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test",
 runtime="nodejs18.x",
 handler="index.handler",
 code={
 "zip_file": """exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
};
""",
 },
 environment={
 "variables": {
 "BUCKET_NAME": my_bucket.bucket,
 },
 })
pulumi.export("lambdaFunctionArn", my_lambda_function.arn) # Directly export the value, no need for a dictionary
pulumi.export("s3BucketName", my_bucket.bucket)

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Aws = Pulumi.Aws;
using AwsCloudControl = Pulumi.AwsNative;

return await Deployment.RunAsync(() =>
{
 // Create an S3 bucket
 var myBucket = new Aws.S3.Bucket("myBucket");

 // Create a Lambda function using AWS Cloud Control
 var myLambdaFunction = new AwsCloudControl.Lambda.Function("myLambdaFunction", new()
 {
 FunctionName = "my-lambda-function",
 Role = "arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test",
 Runtime = "nodejs18.x",
 Handler = "index.handler",
 Code = new AwsCloudControl.Lambda.Inputs.FunctionCodeArgs
 {
 ZipFile = @"exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log(""Event triggered: "", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
};
",
 },
 Environment = new AwsCloudControl.Lambda.Inputs.FunctionEnvironmentArgs
 {
 Variables =
 {
 { "BUCKET_NAME", myBucket.BucketName },
 },
 },
 });

 // Export the Lambda function ARN and the S3 bucket name directly
 return new Dictionary<string, object?>
 {
 ["lambdaFunctionArn"] = myLambdaFunction.Arn,
 ["s3BucketName"] = myBucket.BucketName,
 };
});


package main

import (
 "github.com/pulumi/pulumi-aws-native/sdk/go/aws/lambda"
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create an S3 bucket using the standard AWS provider
 myBucket, err := s3.NewBucket(ctx, "myBucket", nil)
 if err != nil {
 return err
 }
 // Create a Lambda function using AWS Cloud Control provider
 myLambdaFunction, err := lambda.NewFunction(ctx, "myLambdaFunction", &lambda.FunctionArgs{
 FunctionName: pulumi.String("my-lambda-function"),
 Role: pulumi.String("arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test"),
 Runtime: pulumi.String("nodejs18.x"),
 Handler: pulumi.String("index.handler"),
 Code: &lambda.FunctionCodeArgs{
ZipFile: pulumi.String(`exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
};
`),
 },
 Environment: &lambda.FunctionEnvironmentArgs{
 Variables: pulumi.StringMap{
 "BUCKET_NAME": myBucket.Bucket,
 },
 },
 })
 if err != nil {
 return err
 }
 // Export the Lambda function's ARN so that it can be referenced later
 ctx.Export("lambdaFunctionArn",myLambdaFunction.Arn)
 // Export the S3 bucket name so it can be used in other parts of the infrastructure
 ctx.Export("s3BucketName", myBucket.Bucket)
 return nil
 })
}

name: your-project-name  # Replace with your project name
runtime: yaml

resources:
# Creating an S3 bucket using the standard AWS provider.
 myBucket:
 type: aws:s3/bucket:Bucket
 properties: {}

# Creating a Lambda function using the AWS Cloud Control provider.
 myLambdaFunction:
 type: aws-native:lambda:Function
 properties:
 functionName: my-lambda-function
 role: arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test
 runtime: nodejs18.x
 handler: index.handler
 code:
 zipFile: |
 exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
 };
# Lambda environment variables are set here.
 environment:
 variables:
 BUCKET_NAME: ${myBucket.bucket}

# Outputs section - Pulumi will output these values after deployment.
outputs:
 lambdaFunctionArn:
 value: ${myLambdaFunction.arn}
 s3BucketName:
 value: ${myBucket.bucket}

Leveraging AWS Cloud Control Provider for Advanced WAFv2 Configurations

The AWS Cloud Control Provider offers greater flexibility for handling complex configurations that may be challenging or impossible with the standard AWS Provider. For example, when configuring WAFv2, you can define arbitrary nested rules, which enables more advanced and customized security logic. With the Cloud Control Provider, you can compose these rules with reusable building blocks, creating configurations that are more modular and scalable. This level of granularity allows for more precise control over web traffic filtering, such as combining multiple rule sets, applying conditions across different layers of the request, and creating highly tailored security policies. This flexibility is particularly useful for organizations with complex security requirements that need to go beyond what the standard AWS Provider can offer.

For example, this configuration involves six levels of nesting rules but can be composed from reusable building blocks:

import * as pulumi from "@pulumi/pulumi";
import * as awscc from "@pulumi/aws-native";

type RuleGroupStatement = awscc.types.input.wafv2.RuleGroupStatementArgs;

// Create an AWS WAFv2 IP Set
const exampleIPSet = new awscc.wafv2.IpSet("exampleIPSet", {
 addresses: ["1.2.3.4/32", "5.6.7.8/32"],
 ipAddressVersion: "IPV4",
 scope: "REGIONAL",
});

const ipMatch: RuleGroupStatement = {
 ipSetReferenceStatement: {
 arn: exampleIPSet.arn,
 },
};

const testURIMatch: RuleGroupStatement = {
 byteMatchStatement: {
 searchString: "/test",
 fieldToMatch: {
 uriPath: {},
 },
 textTransformations: [
 {
 priority: 0,
 type: "NONE",
 },
 ],
 positionalConstraint: "EXACTLY",
 },
};

const hostMatch: RuleGroupStatement = {
 byteMatchStatement: {
 fieldToMatch: {
 singleHeader: {
 name: "host",
 },
 },
 positionalConstraint: "EXACTLY",
 searchString: "example.com",
 textTransformations: [
 {
 type: "NONE",
 priority: 0,
 },
 ],
 },
};

const internalLabelMatch: RuleGroupStatement = {
 labelMatchStatement: {
 scope: "LABEL",
 key: "internal",
 },
};
const xssMatch: RuleGroupStatement = {
 xssMatchStatement: {
 fieldToMatch: {
 body: {},
 },
 textTransformations: [
 {
 priority: 0,
 type: "NONE",
 },
 ],
 },
};

function matchNot(statement: RuleGroupStatement): RuleGroupStatement {
 return { notStatement: { statement } };
}

function matchAll(statements: RuleGroupStatement[]): RuleGroupStatement {
 return { andStatement: { statements: statements } };
}

function matchAny(statements: RuleGroupStatement[]): RuleGroupStatement {
 return { orStatement: { statements: statements } };
}

// Create an AWS WAFv2 Rule Group
const exampleRuleGroup = new awscc.wafv2.RuleGroup("exampleRuleGroup", {
 capacity: 2000,

 rules: [
 {
 action: { allow: {} },
 name: "rule-1",
 priority: 1,
 statement: matchAny([
 matchAll([ipMatch, xssMatch]),
 matchNot(
 matchAny([
 testURIMatch,
 matchAll([
 internalLabelMatch,
 matchNot(matchAll([ipMatch, hostMatch])),
 ]),
 ]),
 ),
 ]),

 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-rule-metric-name",
 sampledRequestsEnabled: true,
 },
 },
 ],
 scope: "REGIONAL",
 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-metric-name",
 sampledRequestsEnabled: true,
 },
});

// Create an AWS WAFv2 WebACL
const exampleWebACL = new awscc.wafv2.WebAcl("exampleWebACL", {
 defaultAction: { allow: {} },
 rules: [
 {
 name: "example-rule-1",
 overrideAction: { count: {} },
 priority: 1,
 statement: {
 ruleGroupReferenceStatement: {
 arn: exampleRuleGroup.arn,
 },
 },
 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-rule-metric-name",
 sampledRequestsEnabled: true,
 },
 },
 ],
 scope: "REGIONAL",
 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-metric-name",
 sampledRequestsEnabled: true,
 },
});

Conclusion

The AWS Cloud Control Provider for Pulumi is a major step forward in fully and faithfully supporting the AWS Cloud Control API and its ecosystem in Pulumi. By closely collaborating with AWS, we’ve created a provider that enables Pulumi users to leverage the latest AWS features as soon as they are released - without waiting for upstream provider changes to arrive in the standard Pulumi AWS Provider.

Our goal is to continually improve and simplify the Pulumi experience, empowering developers with the tools they need to manage cloud infrastructure seamlessly. This new provider represents our commitment to delivering cutting-edge capabilities as quickly as possible.

We look forward to your feedback and encourage you to explore the full potential of the Pulumi AWS Cloud Control Provider. Don’t hesitate to reach out via the Pulumi AWS Cloud Control Provider GitHub repository with any questions or suggestions as we continue to refine and enhance this experience.

Introducing: Support For Using Any Terraform Provider with Pulumi

Luke Hoban — Thu, 29 Aug 2024 10:00:00 -0600

For the most up-to-date information about using any Terraform provider with Pulumi, see the Any Terraform Provider documentation.

One of our core goals at Pulumi is to provide access to manage any cloud infrastructure with a single unified programming model. Whether it’s multi-cloud (AWS+Azure+Kubernetes), hybrid cloud (GCP+VMWare+Cisco), or managed services (Databricks+GitHub+Cloudflare), Pulumi makes it easy to deploy and manage infrastructure across all of your cloud environments using any of the 150+ cloud providers in the Pulumi Registry.

We’re excited to take this even further by introducing support for using any Terraform or OpenTofu provider from within your Pulumi programs. If there is a long tail Cloud or SaaS platform that has a provider for those ecosystems, it now works with Pulumi as well. And if your organization has built your own custom Terraform or OpenTofu provider to support an internal cloud platform, you can use it from Pulumi as well, without having to publish it to any registry.

Terraform providers are the closest thing the industry has today to a “defacto standard” for describing the resource model of a cloud platform. The vast majority of these providers are implemented by cloud providers, partners, or cloud ecosystem contributors - as a way to describe what resources are available in their platform that can be managed via various kinds of infrastructure tooling. These investments become even more impactful when the work done to define the resource model can be widely leveraged by other tools for a variety of purposes - from IaC to Policy to Asset Inventory and more. There is nothing innately Terraform-specific about these providers - they are simply a standard resource model and the implementation of CRUD (Create-Read-Update-Delete) operations against that model.

At Pulumi, we’ve supported bridging Terraform providers for use in Pulumi from the beginning. Pulumi’s provider model is even more expressive than Terraform’s and we have a defined way to map Terraform providers (as well as other provider ecosystems) into the Pulumi provider model to be used by Pulumi’s CLI and deployment orchestration engine. Many of the most used providers in the Pulumi ecosystem are bridged Terraform providers. However, until now, bridging a new provider required a decent amount of up front and ongoing work, to maintain a separate repository, build and push separate releases, and publish resulting SDKs to every language’s package manager. This meant that while the largest and most used Terraform providers were available in Pulumi, the long tail of providers were not as easily accessible. With this launch, we are now opening up easy access to all of these providers.

Using any Terraform or OpenTofu provider takes just a few seconds using a single command:

$ pulumi package add terraform-provider <your-terraform-provider>

This is enabled by our new Local Packages feature, which lets you generate Pulumi Packages locally into your Pulumi project, instead of relying only on packages published to the Pulumi Registry. Support for generating local packages is available for all Pulumi languages.

Local packages are available as a convenient alternative to the existing support for authoring Native or Bridged Pulumi Packages. You can read more about all of the options available to authoring and using Pulumi Packages in the Pulumi Packages documentation, or about how to generate a local package for a Terraform or OpenTofu provider in the Any Terraform Provider docs.

Walkthrough

Imagine you are working on a new cloud infrastructure project, and come across a technology you want to adopt, but don’t yet see a Pulumi provider in the Pulumi Registry. What do you do?

Let’s walk through a concrete example. We’ll work with a Python project:

$ pulumi new python

Planetscale is an exciting new fully-managed MySQL database platform. Although there isn’t yet an official Pulumi provider for Planetscale in the Pulumi Registry, we can easily use Planetscale from Pulumi if we want to. First, we generate a local package using the pulumi package add command.

$ pulumi package add terraform-provider planetscale/planetscale
Successfully generated a Python SDK for the planetscale package at /Users/lukehoban/github/lukehoban/anytftest/sdks/planetscale

To use this SDK in your Python project, run the following command:

 echo sdks/planetscale >> requirements.txt

 pulumi install

You can then import the SDK in your Python code with:

 import pulumi_planetscale as planetscale

We ask to use the terraform-provider package, which implements support for any Terraform or OpenTofu provider, and then ask specifically for support for the planetscale/planetscale provider. By default, this is pulled from the OpenTofu registry, but you can also specify a local path to the provider, or a fully-qualified reference to any Terraform Registry API-compatible server.

As we see from the output, this generates a local ./sdks/planetscale folder with a Python SDK for using the Planetscale provider.

After running the two commands provided in the output, we now have the local SDK incorporated into our project:

$ echo sdks/planetscale >> requirements.txt
$ pulumi install

We can add some code that uses this new SDK:

import pulumi
import pulumi_planetscale as planetscale

db = planetscale.Database(
 resource_name="mydb",
 organization="lukehoban",
 cluster_size="PS-10"
)

pulumi.export("db_url", db.url)

We get strong typing, completion lists, and error reporting over this SDK from within our IDE.

Finally, we can deploy our code, deploying a new Planetscale database directly from Pulumi:

$ pulumi up
Updating (luke-pulumi-corp/dev2)

View in Browser (Ctrl+O): https://app.pulumi.com/luke-pulumi-corp/anytftest/dev2/updates/2

 Type Name Status
 pulumi:pulumi:Stack anytftest-dev2
 + └─ planetscale:index:Database mydb created (1s)

Outputs:
 + db_url: "https://api.planetscale.com/v1/organizations/lukehoban/databases/mydb-77af236"

Resources:
 + 1 created
 1 unchanged

Duration: 3s

Clicking on the link, we see our new managed database ready for connections! Within just a few seconds, we’ve incorporated support for this new service into our Pulumi infrastructure!

Planetscale is an exciting example, but it is just one of thousands of 3rd party (published in a public registry) and 2nd party (published internally within an organization) providers that are now available for use within Pulumi.

Why Support All Terraform Providers?

Pulumi has a rich and expressive Native Provider model for building providers directly against the full capabilities of the Pulumi Infrastructure as Code resource model. Many of the most important Pulumi providers are built directly on this foundation, including our Azure Native and Kubernetes providers. So why are we excited to support using Terraform and OpenTofu providers from Pulumi?

In short, we believe in giving Pulumi users access to every ecosystem of great providers and components available across the industry and in meeting partners where they are. If a partner or user has built something exciting for managing infrastructure in the cloud, we want Pulumi users to benefit from it. Similarly, we want the partner or user to easily extend the value of their provider to also be able to reach the Pulumi user base, and all of the benefits that Pulumi can offer for Infrastructure as Code management, with no extra effort on their part.

We’ve embraced this approach in many other areas as well:

Pulumi enables deploying and managing the lifecycle of any Kubernetes YAML file or Helm Chart. There’s a great ecosystem and library of Helm charts, and we make those all available to Pulumi users without any wrappers or indirections.
We introduced the Pulumi AWS CDK Adapter to enable Pulumi users to incorporate AWS CDK constructs directly into their Pulumi programs without rewriting in Pulumi first.
And, of course, Pulumi offers features for deploying and managing Azure Resource Manager templates and CloudFormation templates, enabling easy deployment of existing cloud provider IaC artifacts.

With the new support for Any Terraform Provider, we’re bringing this same experience to the Terraform and OpenTofu provider ecosystems.

Use any Terraform provider

Add any Terraform or OpenTofu provider to your Pulumi program with a single command, and manage it alongside the rest of your cloud infrastructure in one workflow.

Learn more

How Does This Work?

Two new technologies in the Pulumi open source project underpin support for using any Terraform or OpenTofu provider.

Parameterized Providers offer support for building a single provider that can implement many different “flavors.” For example, the terraform-provider provider can be parameterized to provide support for any Terraform or OpenTofu provider.

Local Packages offer the ability to generate a Pulumi SDK for the language of your Pulumi project into a local folder, which can be directly incorporated into your project via your existing package manager along with pulumi install.

Local packages are versioned when you include a version in the pulumi package add command, and parameterized providers support many different formats for their parameters. For example, the most general form of the command to use an existing Terraform provider is:

$ pulumi package add terraform-provider [<registry>/]<author>/<name> [version]

Or:

$ pulumi package add terraform-provider /path/to/my/terraform-provider-binary

What’s Next?

Today’s release is a big step in opening up access to even more providers for Pulumi users. But there is still a lot more we have planned to build on top of these new foundations.

First, with locally generated SDKs, the documentation is available within the language SDK you generate (and so lights up in your IDE or development environment!). Still, there is no static docs page available in the Pulumi Registry to find and link to on the web. In the future, we plan to offer the ability to publish documentation for locally bridged providers in the Registry. (tracked in registry#5302) In that case, the registry entry will be only the docs - the SDKs and provider binary will not need to be published/hosted in the Registry itself.

Second, the underlying Local Packages and Parameterized Providers technologies for building different flavors of a single provider are applicable to many other use cases. In addition to having a single provider that can be “parameterized” to act as any Terraform provider, we have plans to apply this technology to additional use cases, such as generating strongly typed providers over any Kubernetes CRD, any Helm Chart, any other Pulumi program, and many more ecosystems of exciting infrastructure components which can be deployed via Pulumi.

Take the new support for using any Terraform or OpenTofu provider from Pulumi for a spin today: