Pulumi Blog: Tejitha Raju

Pulumi AWS Cloud Control Provider is Generally Available

Tejitha Raju — Tue, 01 Oct 2024 09:00:00 -0700

We’re excited to announce the general availability of the new AWS Cloud Control Provider for Pulumi (previously “AWS Native”). AWS is one of the most-used cloud providers across the Pulumi ecosystem, and we are committed to providing the fastest possible access to new AWS capabilities to all Pulumi users. This provider delivers on that promise by providing native support for all resources in the AWS Cloud Control APIs. As part of this launch, we are also renaming the provider from “AWS Native” to “AWS Cloud Control Provider,” based on customer feedback during the preview phase, to provide users with a broader range of advanced cloud management tools exposed by the AWS Cloud Control API. This provider includes the following features and benefits,

Day 1 support for AWS Resources: Provides full coverage of the AWS Cloud Control API, typically on the day of the launch, ensuring users can adopt new features immediately.
Complementary Pulumi Experience: Works alongside Pulumi’s AWS Provider, enhancing Infrastructure as Code projects with the latest AWS capabilities without requiring significant changes to existing setups.
Extended support for third-party resources: Our platform offers enhanced flexibility and interoperability by supporting third-party resources available in the CloudFormation Registry, such as Atlassian, MongoDB, Datadog, and more.
Easy CloudFormation to Pulumi Migration: cf2pulumi provides the ability to migrate existing CloudFormation templates into Pulumi programs in your favorite language.

At Pulumi, we’re dedicated to empowering our customers with the tools they need to innovate and thrive in the cloud. The launch of the AWS Cloud Control Provider represents our unwavering commitment to providing best-in-class solutions that simplify cloud management, streamline operations, and drive business success. The Pulumi AWS Cloud Control Provider can be used side-by-side with the standard Pulumi AWS provider and nearly 200 additional Pulumi resource providers that cover a wide variety of cloud and SaaS platforms.

Using Pulumi AWS Cloud Control Provider with the Pulumi AWS Provider

Let’s walk through an example of using the AWS Cloud Control Provider alongside the Pulumi’s standard AWS Provider. This example shows how to create an S3 bucket with the standard AWS provider and deploy a Lambda function using the AWS Cloud Control provider. The Lambda function is set up to use the S3 bucket’s name and process events accordingly. This example showcases how both providers can work together, leveraging the standard AWS Provider for widely supported resources, while using AWS Cloud Control for the latest features and APIs provided by AWS.

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awscc from "@pulumi/aws-native";

// ARN of the IAM role that the Lambda function will assume
const roleArn = "arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test";

// Create an S3 bucket using the AWS provider
const bucket = new aws.s3.Bucket("myBucket");

// Create the AWS Lambda function using the AWS Cloud Control provider
const lambdaFunction = new awscc.lambda.Function("myLambdaFunction", {
 functionName: "my-lambda-function",
 role: "arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test",
 runtime: "nodejs18.x",
 handler: "index.handler",
 code: {
 zipFile: `
 exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
 };
 `,
 },
 environment: {
 variables: {
 BUCKET_NAME: bucket.bucket,
 },
 },
});

// Export the Lambda function ARN and the S3 bucket name
export const lambdaFunctionArn = lambdaFunction.arn;
export const s3BucketName = bucket.bucket;

import pulumi
import pulumi_aws as aws
import pulumi_aws_native as aws_cloudcontrol
# Create an S3 bucket using the standard AWS provider
my_bucket = aws.s3.Bucket("myBucket")

# Create the Lambda function using the AWS Cloud Control provider
my_lambda_function = aws_cloudcontrol.lambda_.Function("myLambdaFunction",
 function_name="my-lambda-function",
 role="arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test",
 runtime="nodejs18.x",
 handler="index.handler",
 code={
 "zip_file": """exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
};
""",
 },
 environment={
 "variables": {
 "BUCKET_NAME": my_bucket.bucket,
 },
 })
pulumi.export("lambdaFunctionArn", my_lambda_function.arn) # Directly export the value, no need for a dictionary
pulumi.export("s3BucketName", my_bucket.bucket)

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Aws = Pulumi.Aws;
using AwsCloudControl = Pulumi.AwsNative;

return await Deployment.RunAsync(() =>
{
 // Create an S3 bucket
 var myBucket = new Aws.S3.Bucket("myBucket");

 // Create a Lambda function using AWS Cloud Control
 var myLambdaFunction = new AwsCloudControl.Lambda.Function("myLambdaFunction", new()
 {
 FunctionName = "my-lambda-function",
 Role = "arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test",
 Runtime = "nodejs18.x",
 Handler = "index.handler",
 Code = new AwsCloudControl.Lambda.Inputs.FunctionCodeArgs
 {
 ZipFile = @"exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log(""Event triggered: "", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
};
",
 },
 Environment = new AwsCloudControl.Lambda.Inputs.FunctionEnvironmentArgs
 {
 Variables =
 {
 { "BUCKET_NAME", myBucket.BucketName },
 },
 },
 });

 // Export the Lambda function ARN and the S3 bucket name directly
 return new Dictionary<string, object?>
 {
 ["lambdaFunctionArn"] = myLambdaFunction.Arn,
 ["s3BucketName"] = myBucket.BucketName,
 };
});


package main

import (
 "github.com/pulumi/pulumi-aws-native/sdk/go/aws/lambda"
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create an S3 bucket using the standard AWS provider
 myBucket, err := s3.NewBucket(ctx, "myBucket", nil)
 if err != nil {
 return err
 }
 // Create a Lambda function using AWS Cloud Control provider
 myLambdaFunction, err := lambda.NewFunction(ctx, "myLambdaFunction", &lambda.FunctionArgs{
 FunctionName: pulumi.String("my-lambda-function"),
 Role: pulumi.String("arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test"),
 Runtime: pulumi.String("nodejs18.x"),
 Handler: pulumi.String("index.handler"),
 Code: &lambda.FunctionCodeArgs{
ZipFile: pulumi.String(`exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
};
`),
 },
 Environment: &lambda.FunctionEnvironmentArgs{
 Variables: pulumi.StringMap{
 "BUCKET_NAME": myBucket.Bucket,
 },
 },
 })
 if err != nil {
 return err
 }
 // Export the Lambda function's ARN so that it can be referenced later
 ctx.Export("lambdaFunctionArn",myLambdaFunction.Arn)
 // Export the S3 bucket name so it can be used in other parts of the infrastructure
 ctx.Export("s3BucketName", myBucket.Bucket)
 return nil
 })
}

name: your-project-name  # Replace with your project name
runtime: yaml

resources:
# Creating an S3 bucket using the standard AWS provider.
 myBucket:
 type: aws:s3/bucket:Bucket
 properties: {}

# Creating a Lambda function using the AWS Cloud Control provider.
 myLambdaFunction:
 type: aws-native:lambda:Function
 properties:
 functionName: my-lambda-function
 role: arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test
 runtime: nodejs18.x
 handler: index.handler
 code:
 zipFile: |
 exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
 };
# Lambda environment variables are set here.
 environment:
 variables:
 BUCKET_NAME: ${myBucket.bucket}

# Outputs section - Pulumi will output these values after deployment.
outputs:
 lambdaFunctionArn:
 value: ${myLambdaFunction.arn}
 s3BucketName:
 value: ${myBucket.bucket}

Leveraging AWS Cloud Control Provider for Advanced WAFv2 Configurations

The AWS Cloud Control Provider offers greater flexibility for handling complex configurations that may be challenging or impossible with the standard AWS Provider. For example, when configuring WAFv2, you can define arbitrary nested rules, which enables more advanced and customized security logic. With the Cloud Control Provider, you can compose these rules with reusable building blocks, creating configurations that are more modular and scalable. This level of granularity allows for more precise control over web traffic filtering, such as combining multiple rule sets, applying conditions across different layers of the request, and creating highly tailored security policies. This flexibility is particularly useful for organizations with complex security requirements that need to go beyond what the standard AWS Provider can offer.

For example, this configuration involves six levels of nesting rules but can be composed from reusable building blocks:

import * as pulumi from "@pulumi/pulumi";
import * as awscc from "@pulumi/aws-native";

type RuleGroupStatement = awscc.types.input.wafv2.RuleGroupStatementArgs;

// Create an AWS WAFv2 IP Set
const exampleIPSet = new awscc.wafv2.IpSet("exampleIPSet", {
 addresses: ["1.2.3.4/32", "5.6.7.8/32"],
 ipAddressVersion: "IPV4",
 scope: "REGIONAL",
});

const ipMatch: RuleGroupStatement = {
 ipSetReferenceStatement: {
 arn: exampleIPSet.arn,
 },
};

const testURIMatch: RuleGroupStatement = {
 byteMatchStatement: {
 searchString: "/test",
 fieldToMatch: {
 uriPath: {},
 },
 textTransformations: [
 {
 priority: 0,
 type: "NONE",
 },
 ],
 positionalConstraint: "EXACTLY",
 },
};

const hostMatch: RuleGroupStatement = {
 byteMatchStatement: {
 fieldToMatch: {
 singleHeader: {
 name: "host",
 },
 },
 positionalConstraint: "EXACTLY",
 searchString: "example.com",
 textTransformations: [
 {
 type: "NONE",
 priority: 0,
 },
 ],
 },
};

const internalLabelMatch: RuleGroupStatement = {
 labelMatchStatement: {
 scope: "LABEL",
 key: "internal",
 },
};
const xssMatch: RuleGroupStatement = {
 xssMatchStatement: {
 fieldToMatch: {
 body: {},
 },
 textTransformations: [
 {
 priority: 0,
 type: "NONE",
 },
 ],
 },
};

function matchNot(statement: RuleGroupStatement): RuleGroupStatement {
 return { notStatement: { statement } };
}

function matchAll(statements: RuleGroupStatement[]): RuleGroupStatement {
 return { andStatement: { statements: statements } };
}

function matchAny(statements: RuleGroupStatement[]): RuleGroupStatement {
 return { orStatement: { statements: statements } };
}

// Create an AWS WAFv2 Rule Group
const exampleRuleGroup = new awscc.wafv2.RuleGroup("exampleRuleGroup", {
 capacity: 2000,

 rules: [
 {
 action: { allow: {} },
 name: "rule-1",
 priority: 1,
 statement: matchAny([
 matchAll([ipMatch, xssMatch]),
 matchNot(
 matchAny([
 testURIMatch,
 matchAll([
 internalLabelMatch,
 matchNot(matchAll([ipMatch, hostMatch])),
 ]),
 ]),
 ),
 ]),

 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-rule-metric-name",
 sampledRequestsEnabled: true,
 },
 },
 ],
 scope: "REGIONAL",
 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-metric-name",
 sampledRequestsEnabled: true,
 },
});

// Create an AWS WAFv2 WebACL
const exampleWebACL = new awscc.wafv2.WebAcl("exampleWebACL", {
 defaultAction: { allow: {} },
 rules: [
 {
 name: "example-rule-1",
 overrideAction: { count: {} },
 priority: 1,
 statement: {
 ruleGroupReferenceStatement: {
 arn: exampleRuleGroup.arn,
 },
 },
 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-rule-metric-name",
 sampledRequestsEnabled: true,
 },
 },
 ],
 scope: "REGIONAL",
 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-metric-name",
 sampledRequestsEnabled: true,
 },
});

Conclusion

The AWS Cloud Control Provider for Pulumi is a major step forward in fully and faithfully supporting the AWS Cloud Control API and its ecosystem in Pulumi. By closely collaborating with AWS, we’ve created a provider that enables Pulumi users to leverage the latest AWS features as soon as they are released - without waiting for upstream provider changes to arrive in the standard Pulumi AWS Provider.

Our goal is to continually improve and simplify the Pulumi experience, empowering developers with the tools they need to manage cloud infrastructure seamlessly. This new provider represents our commitment to delivering cutting-edge capabilities as quickly as possible.

We look forward to your feedback and encourage you to explore the full potential of the Pulumi AWS Cloud Control Provider. Don’t hesitate to reach out via the Pulumi AWS Cloud Control Provider GitHub repository with any questions or suggestions as we continue to refine and enhance this experience.

Announcing the New Pulumi Partner Program

Isaac Harris — Mon, 16 Sep 2024 00:00:00 +0000

We are excited to announce the launch of the Pulumi Partner Program, a strategic initiative designed to empower our partners and their customers to accelerate cloud projects and achieve faster time-to-market. This program is crafted for cloud and SaaS companies who are building, integrating, and going to market with Pulumi.

Hear from dozens of partners and customers who automate, secure and manage everything in the cloud at PulumiUP on Wednesday, September 18. Register here.

Why the Pulumi Partner Program?

Pulumi has always been committed to the open-source community and the vibrant ecosystem that has grown around our products. With nearly 200 integrations today, our ecosystem is thriving, supported by a network of service partners focused on helping customers succeed. The Pulumi Partner Program is being developed to formalize and strengthen these relationships, allowing us to scale our ability to deliver seamless integrations, co-developed solutions, and collaborative go-to-market activities. By building this program, we aim to empower partners to better support our mutual customers, accelerate cloud adoption, and provide tailored solutions that meet the unique needs of each organization.

Customers are choosing Pulumi because it enables them to manage cloud resources and secrets using familiar general-purpose programming languages like Python, TypeScript, .NET, Java and Go. This approach taps into the existing skills of development teams, reducing the learning curve, and also allows for greater flexibility and reuse of code across projects. By utilizing these widely adopted languages, Pulumi increases team productivity, enables more robust automation, and significantly accelerates the path to production compared to tools that require more restrictive, domain-specific languages. With the introduction of local packages, we’re further simplifying the integration process for partners, facilitating seamless adoption and faster delivery of cloud projects.

Key Benefits of Pulumi Partner Program

Empower Your Customers to deliver cloud initiatives faster using Pulumi’s Infrastructure as Code (IaC) and Secrets Management capabilities. By partnering with Pulumi, your customers gain access to best automation for their cloud projects.
Accelerate Adoption of your solutions through seamless integration with Pulumi. Leverage Pulumi’s support for the world’s most popular programming languages and developer tools to streamline integration, making it easier for your customers to use and adopt your products using familiar languages, tools and process.
Reach More Developers with collaborative documentation, shared examples, and co-marketing initiatives to raise awareness. Participate in joint marketing efforts, including co-branded campaigns and events, to connect with more developers and platform engineers.

With 20 inaugural members, our technology partners join an elite community that benefits from Pulumi’s ongoing investment in technical, marketing, training, and business strategy support. We are excited to welcome new members such as Docker, 1Password, Pinecone, NewRelic, Tailscale, Dagger and many others.

Join the Pulumi Partner Program

We believe that the Pulumi Partner Program will bring together and support the growth of our entire ecosystem, leveraging the strengths of our partners and creating connections for new potential services and technologies. Prospective partners can reach out to us via our website. We look forward to welcoming new partners and driving innovation together.

Announcing 1Password Support for Pulumi ESC in Public Preview

Tejitha Raju — Wed, 27 Mar 2024 12:00:00 -0600

Today, we are thrilled to add integration support for 1Password in preview within Pulumi Environments, Secrets, and Configuration (ESC). Since its launch, numerous organizations have leveraged Pulumi ESC to manage secrets and simplify configurations using composable environments. Leveraging Pulumi ESC’s comprehensive set of providers, users have obtained dynamic cloud provider credentials and retrieved secrets from other secrets management platforms, including HashiCorp Vault.

The addition of 1Password, known for its developer-centric approach to secrets management, to our roster of integrations has been a top request among our community. As users of 1Password ourselves, this collaboration represents more than just a feature release; it embodies our commitment to enriching the developer experience and fostering secure, efficient development workflows.

Pulumi ESC’s addition of the 1Password provider empowers development teams to seamlessly incorporate secrets and configurations stored within 1Password into their workflows. By using the retrieved values as environment variables or within Pulumi Config at runtime, teams can bypass the manual and error-prone process of copying and pasting secrets, significantly enhancing security and efficiency.

Join our upcoming workshop, “Managing team secrets with 1Password & Pulumi ESC”, to explore Pulumi ESC and the 1Password integration in detail.

How does it work?

Create a 1Password service account with access to necessary vaults
Enter the service token into the 1Password provider configuration within your Pulumi ESC Environment (See syntax below)
Define a path-name, and enter the vault name, item name and field name that you want to import into the path name you defined

1Password provider syntax:

values:
 1password:
 secrets:
 fn::open::1password-secrets:
 login:
 serviceAccountToken:
 fn::secret: <input your service token here>
 get:
 <path-name>:
 ref: op://<vault-name>/<item-name>/[section-name/]field-name

Demo

Now that we know how to use the Pulumi ESC 1Password provider, let’s use it to pull AWS CLI credentials stored in 1Password and use them within Pulumi ESC.

Create AWS CLI credentials through AWS IAM and store it in 1Password Vault. In the demo, we’ve stored the AWS credentials in Engineering vault with aws-cli-creds item name with two fields access-key and secret-access-key
Create a new Pulumi ESC Environment with the following definition. Ensure to replace the serviceAccountToken with your 1Password token

values:
 1password:
 secrets:
 fn::open::1password-secrets:
 login:
 serviceAccountToken:
 fn::secret: ops_eyJzaWduSW5B..[Redacted]
 get:
 aws-access-key:
 ref: op://Engineering/aws-cli-creds/access-key
 aws-secret-access-key:
 ref: op://Engineering/aws-cli-creds/secret-access-key
 environmentVariables:
 AWS_ACCESS_KEY_ID: ${1password.secrets.aws-access-key}
 AWS_SECRET_ACCESS_KEY: ${1password.secrets.aws-secret-access-key}

Open the environment to ensure we are able to successfully pull the credentials from 1Password
Use the Pulumi ESC run command to run any AWS CLI commands

pulumi env run passes the configuration stored under the environmentVariables section into a temporary environment’s env variables. The secure credentials are never stored locally on your machine.

Unlocking New Possibilities

With 1Password integration available within Pulumi ESC, we unlock a range of opportunities for developers to enhance workflow efficiency and security:

Broad Cloud Provider Support: What we demonstrated with AWS applies equally to other cloud providers.
Database Secrets Management: Securely manage and inject database credentials for PostgreSQL, MySQL, etc., into your infrastructure as code (IaC) projects.
CI/CD Integration: Utilize Pulumi ESC and 1Password within CI/CD pipelines, such as GitHub Actions, to ensure secure and efficient workflows.

These examples merely scratch the surface of what’s possible. The combination of 1Password and Pulumi ESC opens up a wealth of scenarios to streamline operations and secure your infrastructure.

Conclusion

Pulumi ESC’s 1Password integration reiterates our commitment to providing choice and flexibility for developers. The 1Password provider brings the power of 1Password within ESC in a highly secure manner. We are excited to continue to work with our partners at 1Password and evolve our integration.

Your journey towards streamlined, secure cloud infrastructure management begins here. We can’t wait to see what you build with Pulumi ESC.

Visit the 1Password CI/CD Integrations page for links to the Pulumi ESC 1Password provider docs
Visit our Pulumi ESC docs to learn more about Pulumi ESC and its supported integrations

As always, we deeply value your insights. Your feedback is instrumental in helping us refine and enhance our solutions to better align with your needs.