Pulumi Blog: Isaac Harris

Best Infrastructure as Code (IaC) Tools for 2026

Asaf Ashirov — Sun, 05 Jul 2026 00:00:00 +0000

Infrastructure as Code (IaC) has evolved beyond simple automation into a fundamental shift toward applying software engineering practices to infrastructure management. In 2026, leading organizations aren’t just provisioning infrastructure—they’re treating it as software, complete with testing, version control, code reviews, and continuous integration.

Since this post was published, Pulumi has added first-class support for HashiCorp Configuration Language (HCL). You can now write Pulumi programs in HCL directly, alongside general-purpose languages like TypeScript, Python, Go, and C#. To see how it works, see Pulumi HCL.

As infrastructure complexity grows, teams increasingly seek approaches that provide the same developer productivity tools they use for application development. While template-based and domain-specific language approaches serve many use cases effectively, teams with complex requirements or programming backgrounds often find that general-purpose programming languages offer advantages in testing, abstraction, and collaboration.

This comprehensive guide examines the most effective infrastructure as code tools available today, providing detailed analysis of core IaC platforms, complementary tools, and related technologies through the lens of software engineering best practices. Whether you’re starting fresh with IaC or evaluating alternatives to overcome limitations in your current toolchain, we’ll help you navigate this complex landscape and choose solutions that truly bring software engineering to infrastructure.

Ready to try one of these approaches yourself? Get started with Pulumi for free and provision your first resource in minutes, or see Pulumi Neo in action to explore how an AI agent can manage infrastructure changes alongside your team.

What is Infrastructure as Code?

Infrastructure as Code (IaC) is an approach to automating the provisioning and management of infrastructure using software engineering principles, approaches, and tools. Rather than manually configuring servers, networks, and cloud resources through user interfaces or command-line tools, IaC enables you to define your entire infrastructure declaratively through code.

This approach brings the same benefits that have revolutionized software development—version control, automated testing, code reviews, and CI/CD pipelines—to infrastructure management.

What Are Infrastructure as Code Tools?

Infrastructure as Code tools are platforms and frameworks that enable you to define, provision, and manage infrastructure resources through code rather than manual processes. These tools translate your infrastructure definitions into API calls that create, modify, or destroy cloud resources across various providers.

The most effective IaC tools share several key characteristics:

Goal-state focus: Define your desired infrastructure outcome, whether through declarative syntax (like YAML/JSON templates) or imperative languages that express declarative intent
Multi-cloud support: Work across different cloud providers and services
State management: Track the current state of your infrastructure
Preview capabilities: Show what changes will be made before applying them
Idempotency: Safe to run multiple times with consistent results

Why Infrastructure as Code Tools Are Essential

The shift to IaC tools addresses fundamental challenges that manual infrastructure management cannot solve at scale:

Accelerates Deployment Velocity: Teams can provision complex multi-cloud architectures in minutes instead of weeks. This speed enables faster time-to-market and more frequent, reliable deployments.

Enables True Collaboration: Infrastructure becomes code that teams can review, test, and approve together. This collaborative approach reduces errors and ensures knowledge sharing across the organization.

Eliminates Configuration Drift: Manual changes lead to inconsistencies between environments. IaC ensures your production, staging, and development environments remain identical, eliminating the notorious “works on my machine” syndrome for infrastructure.

Provides Cost Control: Automated provisioning and deprovisioning prevents resource sprawl. Teams can easily track infrastructure costs, set budget alerts, and optimize resource usage across environments.

Ensures Compliance and Security: Codified security policies and compliance requirements are automatically enforced across all deployments. Audit trails become automatic, and policy violations are caught before deployment.

Guarantees Business Continuity: Complete infrastructure definitions stored in version control enable rapid disaster recovery. Organizations can reconstruct entire environments from code, minimizing downtime and data loss.

Infrastructure as Code Tools Overview

This guide covers the following infrastructure as code tools and platforms:

10 Most Used IaC Tools in 2026

Pulumi IaC - Modern IaC with general-purpose programming languages
Terraform - BSL-licensed IaC from HashiCorp that uses the HCL domain-specific language
AWS CDK - Cloud Development Kit for AWS
AWS CloudFormation - Native AWS integration
Azure Resource Manager (ARM) - Native Azure JSON templates
Azure Bicep - Azure-native domain-specific language that compiles to ARM
Google Cloud Infrastructure Manager - Terraform-based solution for Google Cloud
Kubernetes YAML - Native Kubernetes resource definitions
Crossplane - Kubernetes as universal control plane
OpenTofu - Open-source Terraform alternative

Configuration Management Tools

Chef - Configuration management and compliance automation
Puppet - Configuration management and compliance automation
Salt - Configuration management and remote execution

Application and Platform Management Tools

Kubernetes - Container orchestration platform

Security and Compliance Tools

Pulumi ESC - Configuration and secrets management platform
Pulumi Insights - Cloud resource search, analytics, and compliance platform
Snyk - Developer security platform with IaC scanning
Wiz - Comprehensive cloud security platform
HashiCorp Sentinel - Policy-as-code framework for Terraform
Checkov - Static analysis for IaC security
TFLint - Terraform linting and validation

Infrastructure Automation and Management Platforms

Pulumi Cloud - Managed service for Pulumi IaC with enterprise features
HashiCorp Cloud Platform - Enterprise SaaS platform for Terraform management and automation
Spacelift - Automation platform for IaC workflows (not an IaC tool itself)
Env0 - Governance and automation platform for existing IaC tools

Core Infrastructure as Code Tools

Here’s how the core IaC tools compare at a glance before we go deep on each one:

Tool	Language / approach	Clouds supported	State management	Best for
Pulumi	Python, TypeScript, JavaScript, Go, .NET, Java, YAML, or HCL	AWS, Azure, Google Cloud, Kubernetes, and 150+ other providers	Pulumi Cloud (managed, free tier available) or self-managed backend	Teams who want flexible, language-agnostic IaC for infrastructure and operations
Terraform	HCL (HashiCorp’s DSL)	AWS, Azure, Google Cloud, and hundreds of community providers	State file, self-managed or via HCP Terraform	Teams with existing Terraform expertise and established workflows
AWS CDK	TypeScript, Python, Java, C#, Go (compiles to CloudFormation)	AWS only	Delegated to the underlying CloudFormation stack	AWS-focused teams who prefer programming languages over templates
AWS CloudFormation	JSON/YAML templates	AWS only	Managed entirely by AWS	AWS-only deployments requiring deep service integration
Azure ARM	JSON templates	Azure only	Managed entirely by Azure	Azure-native deployments requiring comprehensive platform integration
Azure Bicep	Bicep DSL (compiles to ARM JSON)	Azure only	Managed entirely by Azure (via ARM)	Azure deployments requiring improved readability and developer experience
Google Cloud Infrastructure Manager	HCL (Terraform-based)	Google Cloud	Terraform state, managed by Google	Google Cloud Platform deployments using Terraform
Kubernetes YAML	YAML manifests	Any Kubernetes cluster	Kubernetes cluster state (etcd)	Teams managing container-native applications and cloud-native infrastructure
Crossplane	YAML / Kubernetes CRDs	Multi-cloud, orchestrated through Kubernetes	Kubernetes cluster state (etcd)	Kubernetes-first organizations managing multi-cloud infrastructure
OpenTofu	HCL (community-governed Terraform fork)	AWS, Azure, Google Cloud, and the Terraform provider ecosystem	State file, self-managed or via compatible remote backends	Teams seeking an open-source Terraform alternative with community governance

Use the table as a map: each tool links to its full breakdown below, where you’ll find licensing, key features, and the tradeoffs behind each “best for.”

1. Pulumi

License: Apache 2.0
Best For: Teams who want flexible, language-agnostic IaC for infrastructure and operations

Pulumi IaC represents a modern approach to infrastructure as code, fundamentally changing how teams approach infrastructure by enabling the use of general-purpose programming languages like Python, TypeScript, JavaScript, Go, .NET, and Java, plus YAML and HCL for simpler configurations. Unlike tools that force teams to learn proprietary domain-specific languages (DSLs), Pulumi leverages familiar languages and software engineering practices, providing unprecedented flexibility, powerful abstractions, and seamless integration with existing development workflows.

Pulumi’s approach combines the best of both imperative and declarative paradigms: you use imperative programming languages to define your desired infrastructure state, but the Pulumi engine processes this declaratively to determine what changes are needed to achieve your intended outcome.

Key Features:

Universal language support: Use Python, TypeScript, JavaScript, Go, .NET, Java, YAML, or HCL configurations—no new DSL to learn
Any cloud, any architecture: Deploy to AWS, Azure, Google Cloud, Kubernetes, and 150+ other providers
Real programming constructs: Leverage loops, conditionals, functions, classes, packages, and third-party libraries
Superior developer experience: Full IDE support with IntelliSense, debugging, and refactoring
Built-in testing: Unit and integration testing for infrastructure code
Policy as Code: Enforce compliance and security policies with CrossGuard
Component ecosystem: Rich library of reusable infrastructure components

Universal Language Code Examples:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

// Create a VPC with automatic subnets
const vpc = new awsx.ec2.Vpc("main-vpc", {
 cidrBlock: "10.0.0.0/16",
 numberOfAvailabilityZones: 2,
});

// Create an ECS cluster
const cluster = new aws.ecs.Cluster("app-cluster");

// Create an Application Load Balancer
const alb = new awsx.elasticloadbalancingv2.ApplicationLoadBalancer("app-alb", {
 vpcId: vpc.vpcId,
 subnetIds: vpc.publicSubnetIds,
});

// Deploy a containerized application
const service = new awsx.ecs.FargateService("app-service", {
 cluster: cluster.arn,
 taskDefinitionArgs: {
 container: {
 image: "nginx:latest",
 memory: 128,
 ports: [{
 containerPort: 80,
 targetGroup: alb.defaultTargetGroup,
 }],
 },
 },
});

export const vpcId = vpc.vpcId;
export const serviceUrl = alb.loadBalancer.dnsName;

import pulumi
import pulumi_aws as aws
import pulumi_awsx as awsx

# Create a VPC with automatic subnets
vpc = awsx.ec2.Vpc("main-vpc",
 cidr_block="10.0.0.0/16",
 number_of_availability_zones=2)

# Create an ECS cluster
cluster = aws.ecs.Cluster("app-cluster")

# Create an Application Load Balancer
alb = awsx.elasticloadbalancingv2.ApplicationLoadBalancer("app-alb",
 vpc_id=vpc.vpc_id,
 subnet_ids=vpc.public_subnet_ids)

# Deploy a containerized application
service = awsx.ecs.FargateService("app-service",
 cluster=cluster.arn,
 task_definition_args=awsx.ecs.FargateServiceTaskDefinitionArgs(
 container=awsx.ecs.TaskDefinitionContainerDefinitionArgs(
 image="nginx:latest",
 memory=128,
 ports=[awsx.ecs.TaskDefinitionPortMappingArgs(
 container_port=80,
 target_group=alb.default_target_group
 )]
 )
 ))

pulumi.export("vpc_id", vpc.vpc_id)
pulumi.export("service_url", alb.load_balancer.dns_name)

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ecs"
 "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/ec2"
 "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/elasticloadbalancingv2"
 awsxecs "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/ecs"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create a VPC with automatic subnets
 vpc, err := ec2.NewVpc(ctx, "main-vpc", &ec2.VpcArgs{
 CidrBlock: pulumi.String("10.0.0.0/16"),
 NumberOfAvailabilityZones: pulumi.Int(2),
 })
 if err != nil {
 return err
 }

 // Create an ECS cluster
 cluster, err := ecs.NewCluster(ctx, "app-cluster", nil)
 if err != nil {
 return err
 }

 // Create an Application Load Balancer
 alb, err := elasticloadbalancingv2.NewApplicationLoadBalancer(ctx, "app-alb", &elasticloadbalancingv2.ApplicationLoadBalancerArgs{
 VpcId: vpc.VpcId,
 SubnetIds: vpc.PublicSubnetIds,
 })
 if err != nil {
 return err
 }

 // Deploy a containerized application
 _, err = awsxecs.NewFargateService(ctx, "app-service", &awsxecs.FargateServiceArgs{
 Cluster: cluster.Arn,
 TaskDefinitionArgs: &awsxecs.FargateServiceTaskDefinitionArgs{
 Container: &awsxecs.TaskDefinitionContainerDefinitionArgs{
 Image: pulumi.String("nginx:latest"),
 Memory: pulumi.Int(128),
 Ports: awsxecs.TaskDefinitionPortMappingArray{
 &awsxecs.TaskDefinitionPortMappingArgs{
 ContainerPort: pulumi.Int(80),
 TargetGroup: alb.DefaultTargetGroup,
 },
 },
 },
 },
 })
 if err != nil {
 return err
 }

 ctx.Export("vpcId", vpc.VpcId)
 ctx.Export("serviceUrl", alb.LoadBalancer.DnsName())
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Aws = Pulumi.Aws;
using Awsx = Pulumi.Awsx;

return await Deployment.RunAsync(() =>
{
 // Create a VPC with automatic subnets
 var vpc = new Awsx.Ec2.Vpc("main-vpc", new()
 {
 CidrBlock = "10.0.0.0/16",
 NumberOfAvailabilityZones = 2,
 });

 // Create an ECS cluster
 var cluster = new Aws.Ecs.Cluster("app-cluster");

 // Create an Application Load Balancer
 var alb = new Awsx.ElasticLoadBalancingV2.ApplicationLoadBalancer("app-alb", new()
 {
 VpcId = vpc.VpcId,
 SubnetIds = vpc.PublicSubnetIds,
 });

 // Deploy a containerized application
 var service = new Awsx.Ecs.FargateService("app-service", new()
 {
 Cluster = cluster.Arn,
 TaskDefinitionArgs = new Awsx.Ecs.Inputs.FargateServiceTaskDefinitionArgs
 {
 Container = new Awsx.Ecs.Inputs.TaskDefinitionContainerDefinitionArgs
 {
 Image = "nginx:latest",
 Memory = 128,
 Ports = new[]
 {
 new Awsx.Ecs.Inputs.TaskDefinitionPortMappingArgs
 {
 ContainerPort = 80,
 TargetGroup = alb.DefaultTargetGroup,
 },
 },
 },
 },
 });

 return new Dictionary<string, object?>
 {
 ["vpcId"] = vpc.VpcId,
 ["serviceUrl"] = alb.LoadBalancer.Apply(lb => lb.DnsName),
 };
});

name: aws-ecs-example
runtime: yaml
description: An example that deploys an ECS Fargate service with load balancer

resources:
 # Create a VPC with automatic subnets
 main-vpc:
 type: awsx:ec2:Vpc
 properties:
 cidrBlock: "10.0.0.0/16"
 numberOfAvailabilityZones: 2

 # Create an ECS cluster
 app-cluster:
 type: aws:ecs:Cluster

 # Create an Application Load Balancer
 app-alb:
 type: awsx:elasticloadbalancingv2:ApplicationLoadBalancer
 properties:
 vpcId: ${main-vpc.vpcId}
 subnetIds: ${main-vpc.publicSubnetIds}

 # Deploy a containerized application
 app-service:
 type: awsx:ecs:FargateService
 properties:
 cluster: ${app-cluster.arn}
 taskDefinitionArgs:
 container:
 image: "nginx:latest"
 memory: 128
 ports:
 - containerPort: 80
 targetGroup: ${app-alb.defaultTargetGroup}

outputs:
 vpcId: ${main-vpc.vpcId}
 serviceUrl: ${app-alb.loadBalancer.dnsName}

Key Features:

General-purpose language support: Use Python, TypeScript, JavaScript, Go, .NET, Java, YAML, or HCL without learning new DSLs
Software engineering practices: Full IDE support, comprehensive testing frameworks, debugging capabilities
Multi-cloud flexibility: Native cloud provider SDKs with same-day feature access across 150+ providers
Incremental adoption: Migration tools and state integration for gradual transitions
Open source licensing: Apache 2.0 ensures long-term freedom and flexibility

Considerations:

Learning curve: Teams new to programming may prefer template-based approaches initially
Ecosystem maturity: Smaller community compared to more established tools like Terraform
Tool complexity: Advanced features may require more setup than simpler template systems

Organizations like Unity, Snowflake, and Starburst have reported significant productivity improvements (80-90% deployment time reductions) when adopting programming language-based approaches. These improvements typically occur when transitioning from manual processes or basic template systems to automated approaches with comprehensive testing, IDE integration, and code reusability. Results vary based on starting point, team expertise, infrastructure complexity, and specific use cases.

Ready to get started? Experience Pulumi’s programming language approach and see how familiar languages can transform your infrastructure management with comprehensive testing, powerful abstractions, and seamless multi-cloud support.

2. Terraform

License: Business Source License (BSL) 1.1 (Not Open Source)
Best For: Teams with existing Terraform expertise and established workflows

Terraform uses HashiCorp Configuration Language (HCL) to define infrastructure across multiple cloud providers. However, its 2023 licensing change to BSL (no longer open source) and inherent limitations with domain-specific languages create challenges for teams requiring advanced software engineering practices.

Key Features:

Extensive provider ecosystem: Largest collection of community-maintained providers covering virtually every cloud service
Established workflows: Mature plan-and-apply process with extensive tooling and CI/CD integrations
Community and resources: Vast ecosystem of modules, extensive documentation, training materials, and community support
Enterprise adoption: Widely adopted in enterprises with established processes and expertise
Module ecosystem: Rich collection of reusable Terraform modules for common patterns

When Terraform Works Well:

Teams with existing HCL expertise and Terraform investments
Infrastructure patterns that fit well within HCL’s declarative model
Teams comfortable with template-based approaches

Considerations for Complex Scenarios:

Language constraints: HCL’s domain-specific nature can require workarounds for complex logic compared to general-purpose programming languages
Testing approach: Primarily supports integration testing; teams needing comprehensive unit testing may require additional tooling
IDE experience: While improving, HCL tooling provides less comprehensive support than mature programming language ecosystems
State management: Requires manual backend configuration and locking setup for team collaboration

Code Example:

data "aws_availability_zones" "available" {}

resource "aws_vpc" "main" {
 cidr_block = "10.0.0.0/16"

 tags = {
 Name = "main-vpc"
 }
}

resource "aws_subnet" "public" {
 count = 2
 vpc_id = aws_vpc.main.id
 cidr_block = "10.0.${count.index + 1}.0/24"
 availability_zone = data.aws_availability_zones.available.names[count.index]

 tags = {
 Name = "public-subnet-${count.index + 1}"
 }
}

3. AWS Cloud Development Kit (CDK)

License: Apache 2.0
Best For: AWS-focused teams who prefer programming languages over templates

AWS CDK allows you to define AWS infrastructure using familiar programming languages, synthesizing CloudFormation templates for deployment while providing higher-level abstractions. CDK addresses many limitations of traditional template-based approaches by enabling general-purpose programming languages.

Key Features:

General-purpose programming languages: TypeScript, Python, Java, C#, JavaScript support with full IDE integration (Go available in Developer Preview)
AWS-optimized constructs: High-level components encapsulating AWS best practices
Type safety: Compile-time checking and IntelliSense support
CloudFormation reliability: Built on AWS’s proven deployment engine

Notable Limitations:

AWS-only ecosystem: Locked into single cloud provider, limiting multi-cloud strategies
CloudFormation constraints: Inherits template size limits and deployment restrictions
Vendor lock-in: Deep AWS integration makes migration to other clouds challenging
Limited cross-cloud consistency: Teams need different tools and approaches for multi-cloud deployments

CDK represents a significant improvement over CloudFormation templates but constrains organizations to AWS-only infrastructure strategies.

Code Example:

import * as cdk from 'aws-cdk-lib';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as ecs from 'aws-cdk-lib/aws-ecs';
import { Construct } from 'constructs';

export class MyStack extends cdk.Stack {
 constructor(scope: Construct, id: string, props?: cdk.StackProps) {
 super(scope, id, props);

 const vpc = new ec2.Vpc(this, 'MyVpc', {
 maxAzs: 2
 });

 const cluster = new ecs.Cluster(this, 'MyCluster', {
 vpc: vpc
 });

 const taskDefinition = new ecs.FargateTaskDefinition(this, 'TaskDef', {
 memoryLimitMiB: 512,
 cpu: 256
 });

 taskDefinition.addContainer('web', {
 image: ecs.ContainerImage.fromRegistry('nginx:latest'),
 portMappings: [{ containerPort: 80 }]
 });

 new ecs.FargateService(this, 'MyService', {
 cluster: cluster,
 taskDefinition: taskDefinition
 });
 }
}

4. AWS CloudFormation

License: Proprietary (AWS Service)
Best For: AWS-only deployments requiring deep service integration

AWS CloudFormation provides the foundation for infrastructure as code on AWS, offering native integration with all AWS services and deep platform-specific features.

Pulumi Integration: Pulumi provides native AWS providers that offer the same comprehensive AWS service coverage as CloudFormation, with the added benefit of using general-purpose programming languages. You can also import existing CloudFormation stacks into Pulumi for gradual migration or hybrid management approaches.

Key Features:

AWS-native: First-party support for all AWS services
JSON/YAML templates: Declarative resource definitions
Stack management: Organized resource grouping and lifecycle management
Change sets: Preview infrastructure changes before deployment
Service integration: Deep integration with other AWS services

Code Example:

AWSTemplateFormatVersion: '2010-09-09'
Resources:
 MyVPC:
 Type: AWS::EC2::VPC
 Properties:
 CidrBlock: 10.0.0.0/16
 EnableDnsHostnames: true
 Tags:
 - Key: Name
 Value: MyVPC

 MySubnet:
 Type: AWS::EC2::Subnet
 Properties:
 VpcId: !Ref MyVPC
 CidrBlock: 10.0.1.0/24
 AvailabilityZone: !Select [0, !GetAZs '']

5. Azure Resource Manager (ARM)

License: Proprietary (Microsoft Service)
Best For: Azure-native deployments requiring comprehensive platform integration

Azure Resource Manager provides the foundational infrastructure as code solution for Microsoft Azure, offering complete support for Azure services through JSON-based ARM templates. As Azure’s native IaC solution, ARM templates provide the most comprehensive coverage of Azure services and features.

Pulumi Integration: Pulumi’s native Azure providers offer equivalent comprehensive Azure service coverage with general-purpose programming languages. ARM templates can be imported into Pulumi, and you can reference ARM deployments from Pulumi programs for hybrid scenarios.

Key Features:

Azure-native: Complete Azure service coverage with first-party support
JSON templates: Declarative resource definitions in JSON format
Resource groups: Logical organization of related resources
Deployment modes: Complete or incremental deployment options
Comprehensive integration: Deep integration with Azure services and features

Code Example:

{
 "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
 "contentVersion": "1.0.0.0",
 "resources": [
 {
 "type": "Microsoft.Storage/storageAccounts",
 "apiVersion": "2023-01-01",
 "name": "mystorageaccount",
 "location": "[resourceGroup().location]",
 "sku": {
 "name": "Standard_LRS"
 },
 "kind": "StorageV2"
 }
 ]
}

6. Azure Bicep

License: MIT
Best For: Azure deployments requiring improved readability and developer experience

Azure Bicep is a domain-specific language (DSL) that simplifies Azure Resource Manager template authoring. Bicep files compile transparently to ARM templates, providing all the capabilities of ARM with significantly improved syntax and developer experience.

Key Features:

Simplified syntax: Clean, readable syntax compared to JSON ARM templates
ARM compilation: Compiles to ARM templates for deployment, ensuring full compatibility
Type safety: Strong typing and IntelliSense support in development environments
Modular design: Support for modules and code reuse across projects
Azure-native: Complete Azure service coverage through ARM template compilation

Code Example:

@description('Storage Account type')
@allowed([
 'Standard_LRS'
 'Standard_GRS'
 'Standard_ZRS'
 'Premium_LRS'
])
param storageAccountType string = 'Standard_LRS'

@description('Location for the storage account.')
param location string = resourceGroup().location

resource storageAccount 'Microsoft.Storage/storageAccounts@2023-01-01' = {
 name: 'mystorageaccount'
 location: location
 sku: {
 name: storageAccountType
 }
 kind: 'StorageV2'
 properties: {}
}

7. Google Cloud Infrastructure Manager

License: Proprietary (Google Service)
Best For: Google Cloud Platform deployments using Terraform

Google Cloud Infrastructure Manager automates the deployment and management of Google Cloud infrastructure resources using Terraform configurations, representing Google’s modern approach to infrastructure as code. Infrastructure Manager replaces Google Cloud Deployment Manager, which reached end of support on December 31, 2025.

Key Features:

Terraform-based: Uses standard Terraform configurations declaratively
Automated workflows: Handles Terraform init, validate, and apply operations
Version control integration: Supports Git repositories and Cloud Storage
Deployment tracking: Comprehensive metadata storage and logging
Multiple Terraform versions: Flexibility in Terraform version selection
Cloud Build integration: Leverages Google Cloud Build for execution environment
Migration path: Provides upgrade path from legacy Cloud Deployment Manager

Code Example:

# main.tf - Terraform configuration for Infrastructure Manager
resource "google_compute_instance" "vm_instance" {
 name = "my-vm"
 machine_type = "e2-medium"
 zone = "us-central1-a"

 boot_disk {
 initialize_params {
 image = "debian-cloud/debian-11"
 }
 }

 network_interface {
 network = "default"
 access_config {
 // Ephemeral public IP
 }
 }

 metadata = {
 startup-script = "echo Hello from Infrastructure Manager!"
 }
}

output "instance_ip" {
 value = google_compute_instance.vm_instance.network_interface[0].access_config[0].nat_ip
}

8. Kubernetes YAML

License: Apache 2.0
Best For: Teams managing container-native applications and cloud-native infrastructure

Kubernetes YAML manifests represent one of the most widely adopted forms of infrastructure as code, enabling teams to define, version, and manage containerized applications and their supporting infrastructure through declarative configuration files.

Key Features:

Declarative configuration: Define desired state through human-readable YAML files
Native Kubernetes integration: Direct integration with Kubernetes API without additional tools
GitOps compatibility: Version control YAML files for automated deployment workflows
Resource relationships: Define dependencies and relationships between Kubernetes resources
Extensive ecosystem: Rich ecosystem of operators and custom resources extending functionality

Code Example:

apiVersion: apps/v1
kind: Deployment
metadata:
 name: web-app
 labels:
 app: web-app
spec:
 replicas: 3
 selector:
 matchLabels:
 app: web-app
 template:
 metadata:
 labels:
 app: web-app
 spec:
 containers:
 - name: web-app
 image: nginx:1.21
 ports:
 - containerPort: 80
 resources:
 requests:
 memory: "64Mi"
 cpu: "250m"
 limits:
 memory: "128Mi"
 cpu: "500m"
---
apiVersion: v1
kind: Service
metadata:
 name: web-app-service
spec:
 selector:
 app: web-app
 ports:
 - port: 80
 targetPort: 80
 type: LoadBalancer

9. Crossplane

License: Apache 2.0
Best For: Kubernetes-first organizations managing multi-cloud infrastructure

Crossplane is a Cloud-Native Framework for Platform Engineering that extends Kubernetes to help organizations build custom infrastructure management platforms, allowing teams to provision and manage cloud resources using Kubernetes APIs and patterns.

Pulumi Integration: Pulumi offers the Pulumi Kubernetes Operator (PKO) that provides similar Kubernetes-native infrastructure management capabilities, plus support for YAML-based definitions. Teams can also use Pulumi programs to provision the underlying infrastructure that Crossplane manages, creating layered infrastructure management approaches.

Key Features:

Kubernetes-native: Uses CRDs and standard Kubernetes patterns
Composite resources: Create higher-level infrastructure abstractions
GitOps compatibility: Seamless integration with GitOps workflows
Multi-cloud support: Provision resources across cloud providers
Policy integration: Leverage Kubernetes RBAC and admission controllers

Code Example:

apiVersion: ec2.aws.crossplane.io/v1alpha1
kind: VPC
metadata:
 name: sample-vpc
spec:
 cidrBlock: 10.0.0.0/16
 region: us-east-1
 tags:
 Name: sample-vpc
 providerConfigRef:
 name: aws-provider-config

10. OpenTofu

License: Mozilla Public License 2.0
Best For: Teams seeking an open-source Terraform alternative with community governance

OpenTofu emerged as a fork of Terraform v1.5.x following HashiCorp’s license change, maintained by the Linux Foundation. It provides high, but not full, compatibility with Terraform while ensuring long-term open-source availability under MPL 2.0 licensing.

Key Features:

True open source: MPL 2.0 license with community governance via Linux Foundation ensuring long-term accessibility
Terraform compatibility: Largely maintains existing workflows, modules, and provider ecosystem, with some divergence emerging as the projects evolve independently
Community-driven development: Transparent roadmap, open contribution process, and vendor-neutral governance
License certainty: Removes concerns about future licensing restrictions for commercial use

When OpenTofu Makes Sense:

Organizations requiring guaranteed open-source licensing for compliance or philosophical reasons
Teams with significant Terraform investments wanting to avoid vendor lock-in
Environments where community governance and transparency are priorities
Projects needing long-term license stability

Architectural Considerations: As a Terraform fork, OpenTofu inherits the same architectural approach, which means teams evaluating it should consider whether HCL-based infrastructure definition meets their long-term needs for testing, IDE integration, and developer productivity, or whether a programming language-based approach might better serve complex scenarios.

Code Example:

resource "aws_instance" "web" {
 ami = "ami-0c55b159cbfafe1d0"
 instance_type = "t3.micro"

 tags = {
 Name = "HelloWorld"
 }
}

output "instance_ip" {
 value = aws_instance.web.public_ip
}

Ansible

License: GPL v3
Best For: Configuration management with some infrastructure provisioning capabilities

Ansible is primarily a configuration management tool, not a pure Infrastructure as Code tool. While Ansible can provision some cloud resources, its core strength lies in configuring and managing software on existing systems rather than comprehensive infrastructure provisioning.

Ansible provides configuration management and limited infrastructure provisioning through its agentless architecture and simple YAML-based playbooks.

Pulumi Integration: Rather than competing with Ansible, Pulumi complements it perfectly. Use Pulumi for infrastructure provisioning and Ansible for configuration management. Pulumi’s Command provider can execute Ansible playbooks as part of your infrastructure deployment, and many Pulumi customers use both tools together for comprehensive infrastructure automation. See example: Deploy WordPress to AWS using Pulumi and Ansible.

Key Features:

Agentless architecture: No software installation required on target systems
YAML playbooks: Human-readable automation definitions
Idempotent operations: Safe to run multiple times
Large module library: Extensive built-in functionality for various systems
Push-based execution: Centralized control and execution

Code Example:

---
- name: Provision AWS infrastructure
 hosts: localhost
 tasks:
 - name: Create VPC
 amazon.aws.ec2_vpc_net:
 name: ansible-vpc
 cidr_block: 10.0.0.0/16
 region: us-east-1
 tags:
 Environment: production
 register: vpc

 - name: Create subnet
 amazon.aws.ec2_vpc_subnet:
 vpc_id: "{{ vpc.vpc.id }}"
 cidr: 10.0.1.0/24
 region: us-east-1
 tags:
 Name: ansible-subnet

Chef

License: Apache 2.0
Best For: Complex configuration management scenarios requiring programmable logic

Chef is a configuration management tool, not an Infrastructure as Code tool. Chef focuses on configuring and maintaining software, services, and system settings on existing infrastructure rather than provisioning cloud resources.

Chef provides configuration management and system automation using Ruby-based recipes and cookbooks, offering powerful programmability for complex configuration scenarios.

Key Features:

Ruby DSL: Full programming language for configuration logic
Agent-based architecture: Continuous compliance and drift detection
Cookbook ecosystem: Reusable configuration patterns and community recipes
Test Kitchen: Infrastructure testing and validation framework
Enterprise features: Advanced reporting and compliance capabilities

Code Example:

# cookbook/recipes/default.rb
package 'nginx' do
 action :install
end

service 'nginx' do
 action [:enable, :start]
end

template '/etc/nginx/sites-available/default' do
 source 'default.erb'
 owner 'root'
 group 'root'
 mode '0644'
 notifies :restart, 'service[nginx]'
end

Puppet

License: Apache 2.0
Best For: Enterprise environments requiring strong governance and compliance

Puppet is primarily a configuration management tool, not a pure Infrastructure as Code tool. Puppet specializes in maintaining desired configuration state on existing systems and ensuring compliance, rather than provisioning cloud infrastructure.

Puppet offers enterprise-grade configuration management with a focus on compliance, governance, and declarative system state management.

Key Features:

Declarative language: Puppet DSL for describing desired system state
Compliance reporting: Built-in governance and audit capabilities
Forge marketplace: Community modules and enterprise content
Enterprise support: Professional services and enterprise features
Continuous enforcement: Ongoing configuration compliance monitoring

Code Example:

# manifests/webserver.pp
class webserver {
 package { 'nginx':
 ensure => installed,
 }

 service { 'nginx':
 ensure => running,
 enable => true,
 require => Package['nginx'],
 }

 file { '/var/www/html/index.html':
 ensure => file,
 content => '<h1>Hello from Puppet!</h1>',
 owner => 'www-data',
 group => 'www-data',
 mode => '0644',
 }
}

Salt

License: Apache 2.0
Best For: Python-oriented teams requiring high-performance configuration management

Salt is primarily a configuration management and remote execution tool, not a pure Infrastructure as Code tool. While Salt can manage some infrastructure components, its primary focus is on configuring systems and executing commands across large infrastructures.

Salt provides fast, scalable configuration management and remote execution using Python, designed for high-performance system automation at scale.

Key Features:

Python-based: Leverage the Python ecosystem and libraries
High performance: Fast execution across large infrastructures
Event-driven automation: Reactive automation and orchestration
Pillar data system: Secure, hierarchical data management
Flexible communication: Support for various communication patterns

Code Example:

# /srv/salt/webserver.sls
nginx:
 pkg.installed: []
 service.running:
 - enable: True
 - require:
 - pkg: nginx

/var/www/html/index.html:
 file.managed:
 - source: salt://files/index.html
 - user: www-data
 - group: www-data
 - mode: 644

Infrastructure as Code Tools

Kubernetes - Container Orchestration Platform

License: Apache 2.0
Best For: Container-native infrastructure and application management

While primarily a container orchestration platform, Kubernetes itself serves as an infrastructure as code tool through its declarative YAML manifests and API-driven resource management.

Key Features:

Declarative configuration: YAML-based resource definitions
API-driven: RESTful API for all resource operations
Self-healing: Automatic recovery and reconciliation
Extensible: Custom resources and controllers
GitOps compatible: Works seamlessly with Git-based workflows

Code Example:

apiVersion: apps/v1
kind: Deployment
metadata:
 name: nginx-deployment
spec:
 replicas: 3
 selector:
 matchLabels:
 app: nginx
 template:
 metadata:
 labels:
 app: nginx
 spec:
 containers:
 - name: nginx
 image: nginx:1.21
 ports:
 - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
 name: nginx-service
spec:
 selector:
 app: nginx
 ports:
 - port: 80
 targetPort: 80
 type: LoadBalancer

Infrastructure as Code Security and Compliance Tools

While the tools above focus on provisioning and managing infrastructure, a complete IaC ecosystem includes security scanning and compliance tools. These tools complement your primary IaC tool by providing security analysis, policy enforcement, and compliance checking:

Security Scanning Tools

Snyk - License: Proprietary
Leading developer security platform that includes comprehensive infrastructure as code scanning alongside container and application security. Provides real-time vulnerability detection, compliance checking, and automated remediation guidance. Integrates with popular development tools and CI/CD pipelines with extensive enterprise adoption.

Wiz - License: Proprietary
Comprehensive cloud security platform that includes infrastructure as code scanning capabilities. Provides vulnerability management, compliance monitoring, and security posture assessment across cloud environments. Offers integration with development workflows and supports multiple IaC formats with strong enterprise presence.

Checkov - License: Apache 2.0
Popular open-source static analysis tool for infrastructure as code that scans cloud infrastructure configurations for security and compliance issues. Supports Terraform, CloudFormation, Kubernetes, Helm, ARM templates, and more. Integrates with CI/CD pipelines and provides over 1000+ built-in policies covering CIS benchmarks, PCI DSS, and GDPR compliance.

Linting and Validation Tools

TFLint - License: MPL 2.0
Terraform linter focused on possible errors, best practices, and security issues in Terraform configurations. Provides pluggable rule sets for cloud providers (AWS, Azure, GCP) and helps enforce coding standards, detect deprecated syntax, and prevent common configuration errors.

These security tools integrate into CI/CD pipelines alongside your chosen IaC tool to provide comprehensive security coverage throughout the infrastructure lifecycle.

Pulumi ESC

License: Apache 2.0 (Open Source) / Proprietary (SaaS)
Best For: Teams needing centralized configuration and secrets management across environments and tools

Pulumi ESC (Environments, Secrets, and Configuration) is a comprehensive platform for managing configuration data, secrets, and environment variables across your entire infrastructure and application stack. ESC provides a single source of truth for configuration that works with any infrastructure tool, not just Pulumi.

Key Features:

Universal configuration management: Works with any infrastructure tool, CI/CD system, or application
Hierarchical environments: Compose configuration from multiple sources with inheritance and overrides
Dynamic secrets: Integration with cloud providers for short-lived credentials and just-in-time access
Policy-based access control: Fine-grained permissions and audit logging for configuration access
Multiple consumption methods: CLI, REST API, SDK integration, and direct cloud provider integration
GitOps workflows: Version control integration with pull request-based configuration management

Code Example:

# Production environment configuration
values:
 app:
 name: myapp
 version: ${fn.fromJSON(aws.ecr.getAuthorizationToken).password}
 database:
 connectionString: ${pulumi.database.connectionString}
imports:
 - shared/common
 - aws/production

Pulumi Insights

License: Proprietary (SaaS)
Best For: Organizations needing comprehensive cloud resource visibility, search, and compliance monitoring

Pulumi Insights provides cloud resource search, analytics, and compliance capabilities across your entire multi-cloud infrastructure, regardless of how resources were provisioned. It offers a unified view of your cloud resources with powerful search, cost analysis, and policy enforcement.

Key Features:

Universal cloud inventory: Discover and catalog resources across AWS, Azure, Google Cloud, and Kubernetes
Advanced search and analytics: Query resources by tags, properties, relationships, and compliance status
Cost optimization insights: Resource cost analysis, trends, and optimization recommendations
Compliance monitoring: Continuous policy evaluation and drift detection across all cloud resources
Resource relationships: Understand dependencies and relationships between cloud resources
Integration with any IaC tool: Works with resources created by Pulumi, Terraform, CloudFormation, or manually

Use Cases:

Cloud governance: Ensure compliance with organizational policies and industry standards
Cost management: Identify unused resources and optimization opportunities
Security auditing: Track resource configurations and detect security misconfigurations
Multi-cloud visibility: Single pane of glass for resources across different cloud providers

Infrastructure Automation and Management Platforms

While the above tools focus on defining and provisioning infrastructure, several platforms provide automation, orchestration, and management capabilities that work with Infrastructure as Code tools. Important: These are not IaC tools themselves, but rather automation platforms that rely on underlying IaC tools.

IaC Automation Platforms

Pulumi Cloud Best For: Teams using Pulumi IaC who need enterprise-grade collaboration, security, and governance

Pulumi Cloud is the smartest and easiest way to automate, secure, and manage everything you run in the cloud. It serves as the managed service companion to Pulumi’s open-source infrastructure as code tool, providing enterprise capabilities that extend beyond basic IaC functionality.

The relationship between Pulumi IaC and Pulumi Cloud follows a Git/GitHub model: Pulumi IaC is like Git (fully open source and functional on its own), while Pulumi Cloud is like GitHub (a managed service that makes the open-source tool much easier to use securely at scale with teams).

Key Features:

Managed state backend: Secure, scalable state management with transactional protocols
Team collaboration: Full audit logs, RBAC, and identity provider integration
Centralized secrets management: Pulumi ESC for configuration and secrets across environments
Cloud inventory and insights: Pulumi Insights for search, compliance, and drift detection
Remote deployments: CI/CD integrations and automated workflows
Policy enforcement: Built-in security and compliance policy as code
Multi-product platform: Integrated IaC, secrets management, and cloud insights

Pulumi IaC works with or without Pulumi Cloud - you can use DIY backends (S3, Azure Blob, etc.) or the managed service. However, Pulumi Cloud becomes the default experience when you install the Pulumi CLI, providing instant collaboration capabilities without the overhead of building and maintaining your own infrastructure management platform.

HashiCorp Cloud Platform (HCP) - License: Proprietary (SaaS)
Best For: Organizations standardizing on HashiCorp tools across infrastructure and security lifecycle management

HashiCorp Cloud Platform (HCP) is an enterprise-grade SaaS platform that provides unified lifecycle management for infrastructure and security operations. HCP Terraform (formerly Terraform Cloud) serves as the managed service for Terraform/OpenTofu workflows, while the broader platform integrates multiple HashiCorp tools.

Key Features:

HCP Terraform management: Workspace organization, remote execution, and state management for Terraform
Infrastructure lifecycle automation: Continuous validation, drift detection, and module lifecycle management
Private VCS access: Secure integration with private version control repositories
Policy enforcement: Sentinel policy-as-code framework for compliance and governance
Integrated security services: HCP Vault for secrets management, HCP Consul for service networking
Enterprise access controls: RBAC, SAML SSO integration, and fine-grained permissions
Module and image management: HCP Packer for automated machine/container image creation

HCP focuses specifically on HashiCorp tool integration and provides a comprehensive platform for organizations that have standardized on the HashiCorp ecosystem for infrastructure and security operations.

Spacelift - Spacelift is not an Infrastructure as Code tool—it’s an automation and workflow platform that relies on other IaC tools like Terraform, OpenTofu, Pulumi, CloudFormation, and Kubernetes. Spacelift provides CI/CD pipelines, policy enforcement, and collaboration features for teams using these underlying IaC tools.

Env0 - Env0 is not an Infrastructure as Code tool—it’s an automation platform that provides workflow management, governance, and collaboration features for existing IaC tools like Terraform, OpenTofu, and Terragrunt. It adds CI/CD pipelines, cost management, and policy enforcement on top of these tools.

Atlantis - An open-source tool that provides GitOps-style workflows for Terraform and OpenTofu by automatically running terraform plan and apply operations via pull request automation.

Development Environment Tools

Vagrant - Vagrant is not an Infrastructure as Code tool—it’s a development environment management tool that creates and configures lightweight, reproducible virtual development environments. While it can provision VMs, its focus is on local development environments rather than cloud infrastructure provisioning.

Docker Compose - While not an IaC tool, Docker Compose defines multi-container applications and can be used alongside IaC tools for application deployment after infrastructure provisioning.

These platforms and tools serve important roles in the infrastructure automation ecosystem but should not be confused with Infrastructure as Code tools themselves. They enhance and orchestrate the work of actual IaC tools rather than replacing them.

The Future of Infrastructure as Code

The infrastructure as code landscape is rapidly evolving toward software engineering maturity, with several transformative trends reshaping how organizations approach infrastructure:

Software Engineering Convergence: The most significant trend is the convergence of infrastructure and software engineering practices. Organizations are moving away from limited DSLs toward full programming languages that enable testing, debugging, refactoring, and other software engineering best practices. This shift enables infrastructure teams to leverage the same tools, skills, and methodologies that have proven successful in application development.

Real-Time Cloud Integration: Native cloud provider SDK integration is becoming the standard, replacing community-maintained providers that lag behind new cloud features. Organizations expect same-day access to new cloud services without waiting weeks or months for provider updates.

Comprehensive Testing Paradigms: Infrastructure testing is evolving beyond basic integration tests to include unit testing, property-based testing, and continuous validation. Teams are applying test-driven development principles to infrastructure, catching issues before deployment rather than discovering them in production.

Internal Developer Platform Evolution: Organizations are building sophisticated Internal Developer Platforms that provide self-service infrastructure capabilities while maintaining governance and compliance. These platforms leverage infrastructure as code tools to create standardized, reusable components that accelerate development velocity.

AI-Enhanced Development: Integration of AI tools to help generate, optimize, and troubleshoot infrastructure code, with particular strength in environments that use familiar programming languages where AI assistance is most mature. Emerging technologies like Model Context Protocol (MCP) and AI prompt templates are beginning to enable more sophisticated AI-infrastructure interactions.

These trends favor tools that embrace software engineering principles from the ground up, rather than attempting to retrofit programming capabilities onto template-based or DSL-limited approaches.

Bring software engineering to IaC

Define infrastructure across AWS, Azure, Google Cloud, and Kubernetes in the language your team already knows, with testing, IDE support, and reusable components.

Get started

Migration and Adoption Strategies

Organizations don’t need to choose between maintaining existing infrastructure and adopting modern IaC approaches. Proven migration strategies enable gradual adoption while preserving operational stability.

Incremental Adoption Approaches

Team-by-Team Migration: Start with new projects or specific teams, allowing gradual skill development and process refinement. Teams can maintain existing infrastructure while building new capabilities with modern tools.

Project-by-Project Transition: Migrate individual applications or services incrementally, enabling teams to learn and optimize approaches before expanding scope.

Hybrid Operations: Use state integration and import tools to reference existing infrastructure while building new components with modern IaC tools.

Proven Migration Timelines

Real-world migrations demonstrate that adoption can be remarkably fast with proper tooling:

Atlassian Bitbucket: Complete Terraform to Pulumi migration in 2 days using automated conversion tools. This rapid migration was enabled by existing Python expertise on the team, well-structured Terraform code, and straightforward infrastructure patterns that converted cleanly.
Enterprise migrations: Typical team migrations complete in weeks, not months, depending on infrastructure complexity and team preparation
Learning curve: Teams with programming experience adapt to language-based IaC approaches within days, while those new to programming may require additional training time

Migration Tools and Resources

Automated Conversion: Tools like tf2pulumi and pulumi convert automatically translate existing infrastructure definitions, preserving logic and structure while upgrading to modern approaches.

State Integration: Import existing cloud resources and reference Terraform state during transition periods, enabling zero-downtime migrations.

Training and Support: Comprehensive documentation, tutorials, and community resources accelerate team onboarding and reduce migration risks.

Best Practices for Successful Adoption

Start with pilot projects to build confidence and establish patterns
Leverage automated migration tools to reduce manual conversion effort
Maintain state integration during transition periods for operational continuity
Invest in team training to maximize productivity gains from modern approaches
Establish testing practices early to realize reliability and quality benefits

The key insight: migration to modern IaC approaches is a technical upgrade, not a complete rebuild. Organizations can preserve existing investments while gaining access to superior tooling and practices.

Frequently Asked Questions

Which IaC tool should I choose for AWS?

For teams wanting programming languages: Pulumi IaC and AWS CDK both offer excellent developer experiences with general-purpose programming languages. Pulumi provides multi-cloud flexibility, while CDK offers deep AWS-native integration.

For AWS-only deployments: CloudFormation provides the deepest native AWS service support and direct integration with AWS services.

For Terraform ecosystem users: OpenTofu provides open-source licensing with Terraform compatibility, while Terraform offers extensive community resources and established workflows.

Is Terraform still worth learning in 2026?

Terraform remains a valuable skill and viable choice for many scenarios, though teams should consider their specific needs and long-term goals:

When Terraform Makes Sense:

Teams building on existing Terraform infrastructure and expertise
Organizations with successful Terraform workflows and established processes
Projects that benefit from Terraform’s extensive module ecosystem
Teams comfortable with HCL and template-based approaches

When to Consider Alternatives:

Teams wanting comprehensive testing capabilities (unit, property, integration testing)
Organizations requiring full IDE support with debugging and refactoring
Projects needing complex programmatic logic or extensive code reuse
Teams with strong programming backgrounds who prefer familiar languages

Key Considerations:

Licensing: Terraform’s Business Source License may create restrictions for some commercial scenarios
Ecosystem: Terraform has the largest provider ecosystem and community resources
Learning investment: Consider whether learning HCL aligns with your team’s skill development goals
Migration options: Tools exist for migrating between different IaC platforms when needs change

The choice often depends on team background, infrastructure complexity, and development workflow preferences rather than one approach being universally superior.

How do I get started with infrastructure as code?

Choose Based on Your Team Profile:

Programming-oriented teams: Consider Pulumi IaC for familiar languages with full IDE support, testing frameworks, and multi-cloud flexibility
Template-oriented teams: OpenTofu or Terraform offer extensive community resources, established workflows, and broad ecosystem support
Cloud-specific teams: Native tools (AWS CDK/CloudFormation, Azure ARM/Bicep, Google Cloud Infrastructure Manager) provide deep platform integration
Kubernetes-focused teams: Consider Crossplane or Pulumi Kubernetes Operator for container-native approaches

Getting Started Steps:

Start small: Begin with simple projects like deploying a single application or basic infrastructure
Use examples: Leverage existing templates, tutorials, and community examples to accelerate learning
Plan for growth: Consider how your tool choice will scale with team size and infrastructure complexity
Experiment: Most tools offer free tiers or trials - try multiple approaches with simple projects to find what fits your team best

What’s the difference between configuration management and infrastructure provisioning?

Infrastructure provisioning (Pulumi IaC, Terraform, CloudFormation) creates and manages cloud resources like VMs, networks, and databases
Configuration management (Ansible, Chef, Puppet) configures and maintains software on existing systems
Many modern tools do both, with Pulumi IaC and Ansible offering comprehensive capabilities across both domains

Can I use multiple IaC tools together?

Absolutely! Many organizations use complementary tools for different aspects of infrastructure management:

Pulumi + Configuration Management:

Pulumi + Ansible: Infrastructure provisioning + server configuration (example)
Pulumi + Chef/Puppet: Cloud resources + complex configuration management

Pulumi + Native Cloud Tools:

Reference existing CloudFormation/ARM deployments from Pulumi programs
Import CloudFormation/ARM resources into Pulumi for gradual migration
Use Pulumi alongside CDK for different parts of AWS infrastructure

Pulumi + Kubernetes:

Pulumi Kubernetes Operator (PKO) for GitOps workflows
Pulumi + Crossplane for layered infrastructure management
Pulumi + security scanners like Checkov or Terrascan for compliance

Terraform + Pulumi Coexistence:

Use Terraform for base infrastructure and Pulumi for Internal Developer Platforms (IDPs)
Reference existing Terraform state from Pulumi programs during gradual migration
Manage different infrastructure layers with different tools based on team expertise

Pulumi IaC is designed for heterogeneous environments where multiple tools may be in use. For example, you can manage existing Terraform or CloudFormation resources with Pulumi, either using both tools in tandem or for temporary management while migrating to native Pulumi IaC code. Other Pulumi tools, like Pulumi IDP, also enable you to manage IaC self-service workflows like other tools on this list. See what’s possible when migrating to Pulumi.

Which tool has the best learning resources?

Pulumi: Excellent documentation with examples in multiple languages, comprehensive tutorials
Terraform/OpenTofu: Extensive community content, many courses and books available
AWS CDK: Outstanding official documentation with workshops and examples
Kubernetes: Vast ecosystem but steeper learning curve

How important is open-source licensing for IaC tools?

Very important for long-term strategy:

Apache 2.0 (Pulumi, CDK) offers maximum flexibility
MPL 2.0 (OpenTofu) ensures open-source availability
BSL (Terraform) restricts commercial competitors
Proprietary (CloudFormation, ARM) ties you to specific vendors

Choose open-source tools like Pulumi or OpenTofu to avoid vendor lock-in.

What about security and compliance?

All major IaC tools support security best practices:

Built-in policy engines: Pulumi CrossGuard, AWS Config, Azure Policy
Third-party scanners: Checkov, KICS, Terrascan work with all tools
Compliance frameworks: Most tools support SOC 2, PCI DSS, CIS benchmarks
Secret management: Integration with Vault, AWS Secrets Manager, Azure Key Vault

Pulumi offers the most comprehensive built-in security features with CrossGuard policy as code.

Can I migrate from one IaC tool to another?

Yes, and many organizations are successfully migrating to overcome limitations in their current tools. Migration tools and proven approaches exist:

Proven Migration Success Stories:

Atlassian Bitbucket: Completed migration from Terraform to Pulumi’s Python-based IaC in just 2 days using migration tools
Starburst: Achieved 112x faster deployments after incrementally migrating from Terraform
Multiple organizations: Report 80-90% deployment time reductions after moving to programming language-based approaches

Available Migration Tools:

Pulumi: Offers pulumi convert for importing from Terraform, ARM, and CloudFormation with state integration
Terraformer: Can import existing cloud resources into Terraform/OpenTofu
CDK Migrate: Helps move from CloudFormation to CDK
Manual migration: Always possible by recreating resources in the new tool

Migration Strategies:

Incremental adoption: Migrate team-by-team or project-by-project without disruption
State integration: Reference existing infrastructure during transition periods
Zero downtime: Tools like Pulumi enable seamless migration without service interruption

The key is choosing tools that provide comprehensive migration support and incremental adoption paths rather than requiring “rip and replace” approaches.

Conclusion: The Evolution of Infrastructure as Code

The infrastructure as code landscape in 2026 reflects a maturing field where different approaches serve different organizational needs and team preferences. The evolution from manual processes to automated infrastructure has branched into multiple viable paths, each with distinct advantages.

The Spectrum of Approaches:

Template-based tools like Terraform and OpenTofu continue to serve teams effectively, particularly those comfortable with declarative configuration and established HCL workflows. Cloud-native solutions provide deep platform integration for single-cloud strategies. Programming language-based approaches offer familiar development experiences for teams seeking comprehensive testing, IDE integration, and code reusability.

Evidence of Success Across Approaches:

Organizations achieve significant improvements regardless of their chosen path—the key is selecting tools that align with team expertise and infrastructure requirements. Understanding what DevOps is and platform engineering concepts helps inform these decisions. Success stories span the entire ecosystem, from Terraform’s widespread enterprise adoption to programming language approaches enabling dramatic productivity gains at companies like Unity, Snowflake, and Starburst.

Choosing Your Path:

The most successful organizations focus on key decision criteria:

Team background: Match tools to existing skills and preferred development approaches
Infrastructure complexity: Consider testing, abstraction, and maintainability needs
Organizational constraints: Factor in licensing, governance, and compliance requirements
Growth trajectory: Plan for how needs may evolve with scale and team expansion
Integration requirements: Ensure compatibility with existing workflows and toolchains

The Future of Infrastructure:

The industry continues evolving toward treating infrastructure as software, but this transformation takes many forms. Organizations exploring serverless architectures and container strategies particularly benefit from programmable infrastructure approaches. Whether through enhanced DSLs, visual design tools, programming languages, or hybrid approaches, the goal remains consistent: enabling teams to manage infrastructure with the same reliability, collaboration, and velocity they expect from modern software development.

For teams ready to embrace programming language-based infrastructure as code, get started with Pulumi for free to experience how familiar languages and software engineering practices can transform infrastructure management with comprehensive testing, powerful abstractions, and seamless multi-cloud support. Want to see where infrastructure is headed next? Explore Pulumi Neo, the AI agent that proposes changes, runs previews, and opens PRs alongside your team.

Announcing the New Pulumi Partner Program

Isaac Harris — Mon, 16 Sep 2024 00:00:00 +0000

We are excited to announce the launch of the Pulumi Partner Program, a strategic initiative designed to empower our partners and their customers to accelerate cloud projects and achieve faster time-to-market. This program is crafted for cloud and SaaS companies who are building, integrating, and going to market with Pulumi.

Hear from dozens of partners and customers who automate, secure and manage everything in the cloud at PulumiUP on Wednesday, September 18. Register here.

Why the Pulumi Partner Program?

Pulumi has always been committed to the open-source community and the vibrant ecosystem that has grown around our products. With nearly 200 integrations today, our ecosystem is thriving, supported by a network of service partners focused on helping customers succeed. The Pulumi Partner Program is being developed to formalize and strengthen these relationships, allowing us to scale our ability to deliver seamless integrations, co-developed solutions, and collaborative go-to-market activities. By building this program, we aim to empower partners to better support our mutual customers, accelerate cloud adoption, and provide tailored solutions that meet the unique needs of each organization.

Customers are choosing Pulumi because it enables them to manage cloud resources and secrets using familiar general-purpose programming languages like Python, TypeScript, .NET, Java and Go. This approach taps into the existing skills of development teams, reducing the learning curve, and also allows for greater flexibility and reuse of code across projects. By utilizing these widely adopted languages, Pulumi increases team productivity, enables more robust automation, and significantly accelerates the path to production compared to tools that require more restrictive, domain-specific languages. With the introduction of local packages, we’re further simplifying the integration process for partners, facilitating seamless adoption and faster delivery of cloud projects.

Key Benefits of Pulumi Partner Program

Empower Your Customers to deliver cloud initiatives faster using Pulumi’s Infrastructure as Code (IaC) and Secrets Management capabilities. By partnering with Pulumi, your customers gain access to best automation for their cloud projects.
Accelerate Adoption of your solutions through seamless integration with Pulumi. Leverage Pulumi’s support for the world’s most popular programming languages and developer tools to streamline integration, making it easier for your customers to use and adopt your products using familiar languages, tools and process.
Reach More Developers with collaborative documentation, shared examples, and co-marketing initiatives to raise awareness. Participate in joint marketing efforts, including co-branded campaigns and events, to connect with more developers and platform engineers.

With 20 inaugural members, our technology partners join an elite community that benefits from Pulumi’s ongoing investment in technical, marketing, training, and business strategy support. We are excited to welcome new members such as Docker, 1Password, Pinecone, NewRelic, Tailscale, Dagger and many others.

Join the Pulumi Partner Program

We believe that the Pulumi Partner Program will bring together and support the growth of our entire ecosystem, leveraging the strengths of our partners and creating connections for new potential services and technologies. Prospective partners can reach out to us via our website. We look forward to welcoming new partners and driving innovation together.

Announcing the Speaker Lineup for PulumiUP 2023

George Huang — Wed, 24 May 2023 00:00:00 +0000

PulumiUP is our virtual user conference for the Pulumi community and anyone interested in Infrastructure as Code. Every year, we strive to build a program packed with technical talks, demos, and best practices with the goal of leaving you with new learnings and inspiration that will help you become a better cloud engineer. You’ll hear from industry leaders and experts about IaC, software engineering, DevOps, Platform engineering, and AI.

We hope you’ll join us virtually and take this opportunity to grow your knowledge and become infrastructure as code stars.

Get Ready for Amazing Launches and Demos

Learn about Pulumi’s newest features across the core IaC SDKs and Pulumi Cloud. And watch hands-on demos by the Pulumi team:

Joe Duffy, Co-founder and CEO
Luke Hoban, Co-creator of TypeScript and CTO
Meagan Cojocar, Principal Product Manager
Evan Boyle, Engineering Manager
Zaid Ajaj, Software Engineer

Final Speakers Announced

You’ll hear from innovators, leaders, and experts from multiple industries and learn about infrastructure as code, software engineering, and cloud engineering.

Sam Cogan - Solution Architect, WTW

Talk: Keeping your Infrastructure Code DRY (Don’t Repeat Yourself)
Sam is a Solution Architect with WTW and a Microsoft Azure MVP. Sam is focused on delivering applications into the cloud using IaaS, PaaS, Serverless, Containers, and Kubernetes. Sam is particularly focused on automation and DevOps, including Infrastructure as Code, Cloud automation tooling, PowerShell, and CI/CD tooling.

Dennis Sauvé - DevOps Engineer, Washington Trust Bank

Talk: How a Bank Modernized its Software Engineering with Infrastructure as Code Automation
Dennis is a development operations engineer at Washington Trust Bank in Spokane, WA. With years of experience in software development and cloud engineering, he works to streamline deployment pipelines and create solutions for developers.

Ala Shiban - Co-Founder & CEO, Klotho

Talk: Creating a ‘Cloud-Aware’ Code Solution
Ala is the co-founder of Klotho, a startup that transforms plain code to cloud-native code. He started his journey on Commodore64 with cassette tapes and (much) more recently worked on distributed systems, developer tools, and large-scale web, cloud, and microservice backends.

He’s led smaller teams and larger organizations at Microsoft’s Developer Division and Riot Games’ Infrastructure Platform organization.

Matt Stephenson - Sr. Principal Software Engineer, Starburst Data

Talk: Creating Infrastructure Automation Magic with Code
Matt is a Sr. Principal Software Engineer at Starburst Data. He’s responsible for Starburst’s infrastructure architecture. He’s worked at many tech companies, including Square, Oracle, Google, and Amazon. Aside from technology, Matt is a pilot and helps build The Man for Burning Man each year.

Tim Holm - Co-Founder & CTO, Nitric

Talk: Engineering a Multi-cloud Platform
Tim Holm is the Co-founder and CTO at Nitric, with 12 years of experience in software engineering and cloud development. In disciplines ranging from Server and Storage Virtualisation and Automation, Mobile Application development, Full-stack application development, and Cloud infrastructure automation. Before founding Nitric, Tim worked to deliver critical infrastructure projects in Government and Fintech.

In his spare time, Tim enjoys video games, building Lego, and mountain biking.

Tyler Scheuble - Head of Platform, People Data Labs

Talk: Using Monorepos to Level-up your Infrastructure as Code and Software Delivery
Tyler possesses a genuine enthusiasm for technology. At People Data Labs, he has designed petabyte-scale data pipelines and high-performance APIs with demanding availability requirements, which informed his approach to leading the development of the next generation of the startup’s development pipelines and environments.

Tyler is passionate about finding simple solutions to cross-cutting concerns such as security, observability, and development velocity.

Sefi Genis - Co-Founder & CTO, Firefly

Talk: Lessons Learned from Writing Thousands of Lines of IaC
Sefi is the CTO and Co-Founder at Firefly. He started his career in the 8200 Unit of the Israeli Intelligence Corps and then continued on to security and developer tooling companies.

Sefi was a founding engineer at Dome9 Security (later acquired by Checkpoint), where he led the Backend Infrastructure of the product post-acquisition. Before founding Firefly, he became Head of Engineering in an Israeli cybersecurity startup company. Today he is building Firefly to help engineers manage the new reality of cloud-native operations.

Panel Discussion - AI and the Future of Cloud Development

Prepare yourself for the future of cloud development! Gain valuable insights from industry experts in our panel discussion, where we explore the transformative impact of AI on the cloud infrastructure and software engineering industries. The panel discussion will be moderated by CEO Joe Duffy. These are the panelists:

Amanda Silver - CVP Developer Tools, Microsoft

Amanda is the CVP of PM for Microsoft’s Developer Division, including the Visual Studio family of products, .NET, TypeScript, and our developer platforms. She has been key to Microsoft’s transformation to contribute to open source with the introduction of TypeScript, Visual Studio Code, and the acquisition of both Xamarin and GitHub. She believes that a tight digital feedback loop with zero distance between end-users and engineering teams is a critical element of great product development.

Beyang Liu - Co-Founder & CTO, Sourcegraph

Beyang is the CTO and co-founder of Sourcegraph. Prior to Sourcegraph, Beyang was a software engineer at Palantir Technologies, where he developed new data analysis software on a small, customer-facing team working with Fortune 500 companies. Beyang studied Computer Science at Stanford, where he published research in probabilistic graphical models and computer vision at the Stanford AI Lab and thoroughly enjoyed his compilers course.

Luke Hoban - TypeScript Co-Creator and CTO, Pulumi

Luke is CTO at Pulumi. He has worked on developer tools his entire career. He started out on Visual Studio, C#, and .NET in the early 2000s, later joined the ECMAScript standards body as a representative of Microsoft and then became one of the co-founders of the TypeScript programming language.

Featured Partners

Thank you to our amazing partners for supporting PulumiUP! Visit these booths for tech talks, demos and Q&A about your favorite cloud platforms and developer tools:

AWS - Solutions Architects will be on hand to demo the latest AWS and Pulumi capabilities and help you get the most from the AWS platform.
Google Cloud - Customer Engineers will help you accelerate your cloud journey with helpful tips and Q&A.
Microsoft - Cloud Advocates will be available to show you how to kickstart your projects on Azure.
Equinix - Stop by the Equinix booth to learn all about the new Equinix Pulumi provider.
CircleCI - Stop by the CircleCI booth to learn best practices for adopting CI/CD in your organization.
New Relic - The New Relic team will demonstrate how to get started with Observability as Code with Pulumi and New Relic.
Firefly - If you have existing cloud resources you can convert them to Pulumi code easily with Firefly. Stop by the Firefly booth for a demo.
Nitric - The Nitric team will be on site to show you how to create cloud-agnostic infrastructure code from your application code.
Klotho - The Klotho team will show you how devops teams an application teams can easily collaborate to moderize and scale infrastructure without disruptions.
Port - Join the Port team to learn how to setup a self-service portal for your developers.

Want to attend PulumiUP 2023? Register now!

AWS Lambda Response Streaming with Pulumi

Isaac Harris — Thu, 06 Apr 2023 00:00:00 +0000

Since its introduction in 2014, AWS Lambda has rapidly expanded its capabilities from simple “functions in the cloud” at launch to a comprehensive serverless platform with support for containerized functions and public per-function URL endpoints.

As serverless applications have increased in sophistication, developers have used functions-as-a-service as a first-class tool in their microservices strategy. As organizations increasingly look to break up their monolithic applications into services, adoption of AWS Lambda has not been a viable option for applications that return payloads larger than the 6 MiB Lambda service limit. In addition to the payload response limit, AWS Lambda has been limited to returning the entire response once it has been fully buffered in memory within the Lambda service. This historical limitation meant that end users would not see any response until the entire response was available. Both of these limitations made Lambda difficult or impossible to use for data-intensive or latency-sensitive use cases.

Introducing Response Streaming

Today, AWS announced Lambda Response Streaming: a new feature that enables response payloads beyond the 6MiB limit, supports binary content in responses and reduces response times for latency-sensitive applications. Response Streaming reduces the time to first byte (TTFB) by enabling your functions to send partial responses to the client as soon as they are ready instead of waiting for a fully-generated and buffered response. This reduces multi-second TTFB to milliseconds, and, you can stream payloads that are gigabytes in size with a soft 20MiB limit for streamed payloads. The maximum throughput for a streamed response is 2MiB/s.

Enabling Response Streaming requires a new Lambda function handler signature that provides a stream object that the function can write to. When your function writes data to this stream, it is sent immediately to the client. To help clients understand the contents of the stream, you can optionally set the Content-Type header for a response.

This feature supports NodeJS v14.x, v16.x and v18.x at launch and comes with an updated billing model that charges for bytes processed and streamed from your Lambda Function.

Authoring a Lambda Function for Response Streaming

For an example of how response streaming works in practice, we’ve built a Lambda function that will stream the Gettysburg Address one sentence at a time with a one second pause between sentences. For brevity, each sentence is abbreviated below (but the code in GitHub, which demonstrates the same function with and without streaming, contains the full text of the speech):

const handleInternal = async (_event, responseStream, _context) => {
 responseStream.setContentType("text/plain");

 const sentences = [
 "Fourscore and seven years ago...",
 "Now we are engaged in a great civil war...",
 "We have come to dedicate...",
 "It is altogether fitting...",
 "But, in a larger sense...",
 "The brave men...",
 "The world will little note...",
 "It is for us the living...",
 ]

 const timer = ms => new Promise( res => setTimeout(res, ms));

 for(let i = 0; i < sentences.length; i++) {
 responseStream.write(`${sentences[i]}\n`);
 await timer(1000);
 }

 responseStream.end();
}

exports.handler = awslambda.streamifyResponse(handleInternal);

The notable changes to the Lambda function code that support response streaming are:

The function must be wrapped in the awslambda.streamifyResponse() middleware. (The streamifyResponse() function is defined in the Node runtime that’s supplied by the Lambda service.)
Handler methods require an additional parameter to the usual (event, context) signature to become (event, responseStream, context).
The responseStream parameter implements Node’s Writable interface to give it a familiar API with other streams that Node developers may be familiar with.
Responses to the client must be written via responseStream.write(), the method that writes to the stream.
The function must call responseStream.end() to signal that no more data is expected.
Note that functions authored for streaming can still be seamlessly invoked with buffered responses.

Streaming Performance Considerations

For this simple text-based example, there is little need to worry about overall performance. However, if the response is large or the client is expected to perform a lot of processing on the response data, AWS suggests using Node’s pipeline() in place of the built-in write() method in case data is produced faster than it can be consumed by the client. pipeline() effectively helps you to pipe between streams, forward errors and properly clean up – providing a callback when the pipeline is complete. You can learn more about this approach in Backpressuring in streams in the Node docs.

Packaging and deploying your Lambda Function for streaming with Pulumi

We can package our Lambda function for streaming with just a few dozen lines of code using Pulumi. First, we’ll need to import some libraries that will allow us to package our Lambda and create the necessary resources in AWS. For this exercise, we’ll be using both the AWS Cloud Control provider and the AWS provider. The AWS Cloud Control provider works with the AWS Cloud Control API to enable same-day support for new AWS features like Lambda response streaming.

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awsNative from "@pulumi/aws-native";

Next, we’ll create an IAM role for our Lambda function to assume that has the standard required permissions for a Lambda function:

const role = new aws.iam.Role("role", {
 assumeRolePolicy: JSON.stringify({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com",
 },
 "Action": "sts:AssumeRole",
 }],
 }),
});

new aws.iam.RolePolicyAttachment("role-policy-attachment", {
 role: role.name,
 policyArn: "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
});

Streamed responses are not currently supported with API Gateway nor with Application Load Balancers, so they must be invoked via Lambda Function URLs or invoked and streamed through the InvokeWithResponseStream API using the AWS SDK. Let’s create our function, add a function URL, and the necessary permissions to allow unauthenticated access to the function URL. Note that we set the invokeMode input of the function URL to RESPONSE_STREAM. This is what allows us to invoke the function with streaming enabled:

const streamingFunc = new aws.lambda.Function("streaming-func", {
 code: new pulumi.asset.FileArchive("../../lambda"),
 role: role.arn,
 handler: "index.handler",
 runtime: "nodejs14.x",
 timeout: 30,
});

new aws.lambda.Permission("streaming-permission", {
 action: "lambda:InvokeFunctionUrl",
 "function": streamingFunc.arn,
 principal: "*",
 functionUrlAuthType: "NONE",
});

const streamingUrl = new awsNative.lambda.Url("streaming-url", {
 authType: "NONE",
 targetFunctionArn: streamingFunc.arn,
 invokeMode: "RESPONSE_STREAM",
});

Finally, we’ll add a stack export so we can access the generated function URL from outside of our Pulumi program:

exports.streamingUrl = streamingUrl.functionUrl;

To deploy our resources, we run pulumi up -y:

$ pulumi up -y
Previewing update (dev)

View Live: https://app.pulumi.com/jkodrofftest/aws-lambda-streaming/dev/previews/d2e04257-222e-4704-b163-98e6bf4ff704

 Type Name Plan Info
 + pulumi:pulumi:Stack aws-lambda-streaming-dev create
 + ├─ aws:iam:Role role create
 + ├─ aws:iam:RolePolicyAttachment role-policy-attachment create
 + ├─ aws:lambda:Function streaming-func create
 + ├─ aws:lambda:Permission streaming-permission create
 + └─ aws-native:lambda:Url streaming-url create


Outputs:
 streamingUrl: output<string>


Updating (dev)

View Live: https://app.pulumi.com/jkodrofftest/aws-lambda-streaming/dev/updates/25

 Type Name Status Info
 + pulumi:pulumi:Stack aws-lambda-streaming-dev creating (43s)...
 + ├─ aws:iam:Role role created (1s)
 + ├─ aws:iam:RolePolicyAttachment role-policy-attachment created (0.33s)
 + ├─ aws:lambda:Function streaming-func created (22s)
 + ├─ aws-native:lambda:Url streaming-url created (2s)
 + └─ aws:lambda:Permission streaming-permission created (0.58s)

Outputs:
 streamingUrl: "https://s5a3pi4zzsufh5cgaigdxq3wva0psaib.lambda-url.eu-west-1.on.aws/"

Resources:
 + 6 created

Duration: 46s

Reading a Response Stream

Now that we’ve deployed our streaming function, we can read a response stream from the deployed Lambda URL. Per AWS, we can use any client that supports HTTP/1.1 chunked transfer coding. In cURL, streamed responses can be read via the -N flag. We can also add output that displays our time to connect, time to first byte (TTFB), and the total time to return the response:

curl -N -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} \n" $(pulumi stack output streamingUrl)

When successful, your output will look like this:

This example visually shows the advantage of response streaming because the first message appears shortly (approximately 1 second) after invocation. If we disable streaming, the fully buffered response takes several seconds to arrive. You can see the buffered response example in action. This is precisely why TFFB is important: it’s the difference between an app or service that appears to hang while awaiting a response and one that feels much more responsive to an end user.

Wrapping up

AWS Lambda Response Streaming continues the Lambda team’s steady drumbeat of innovation that adds new capabilities and use cases to serverless functions. Grab the sample code and give the feature a try. Then, let us know what you think in the Pulumi Community Slack.

AWS Lambda SnapStart with Pulumi

Josh Kodroff — Mon, 28 Nov 2022 14:00:00 -0800

As AWS Lambda has matured as a serverless platform, there are two key ways the service has evolved:

New capabilities that extend the platform to support new use cases like Lambda Container support, Lambda URLs and attribute-based access control support.
Performance enhancements that enable Lambda functions to be more responsive and cost-effective such as Tiered compilation, and Graviton2 support are just a few examples of the investments AWS made in this space.

With each advancement, the reasons to deploy a full VM or cluster to support your application get fewer and the time to deliver value in the cloud gets shorter.

One challenge that many teams have faced when adapting their services to take advantage of AWS Lambda’s simple deployment model and attractive economics is cold starts: the latency incurred when the service prepares a new execution environment for your function. There is good analysis on the source of this latency as well as some available optimizations here.

Functions using Java runtimes typically experience significantly longer initialization times than functions using other runtimes.

Lambda creates a fresh execution environment when a function is invoked for the first time or when the function gets scaled to handle a spike in traffic. Initialization requires downloading function code, initializing the runtime, and running any pre-invocation initialization code in your function code. (Examples of pre-invocation code are retrieving secrets from AWS Secrets Manager or initializing database or other network connections.) This initialization process can add seconds of latency to the first invocation of your function.

Reducing Function Latency with AWS Lambda SnapStart

Today, AWS released a major improvement for Java-based functions that specifically addresses the latency incurred from cold starts in Java functions: AWS Lambda SnapStart. SnapStart improves startup performance for latency-sensitive Java applications by up to 10x at no extra cost, and typically with no changes to your function code. With SnapStart, instead of initializing function code on invoke, initialization happens when a function version is published. The way AWS describes this new feature is that the Lambda service will take a Firecracker MicroVM snapshot of the memory and disk state of the function’s fully-initialized execution environment, persist the encrypted snapshot and place it in a cache it for rapid access. On a first invocation or on scaling the function, Lambda resumes new environments using this snapshot.

AWS Lambda further improves invoke times by dividing snapshots into chunks. This means only the chunks required for your function are downloaded from a distributed cache on invoke. In real terms, this process removes several seconds of latency from function execution and scaling times.

The new Lambda Telemetry API works with SnapStart so you can easily track platform metrics related to this new capability.

All of the benefits of SnapStart are a great reason to give this feature a try, and there is no additional cost for enabling SnapStart for your functions. However, there are some considerations to keep in mind before you get started:

SnapStart improvements require the Amazon Corretto 11 runtime - Amazon’s production-ready distribution of OpenJDK.
After 14 days without an invocation, your function version becomes inactive and Lambda will delete the snapshot. If you invoke an inactive function version, Lambda will initialize a new snapshot.
Network connections may not persist when a function is invoked from a snapshot, so, your code should assume connections need to be restored.
Ephemeral data such as temporary credentials created during initialization may expire between the time the snapshot is created and later resumed.
Since snapshots are reused many times to resume execution environments, there are additional requirements to ensure random numbers generated by your function remain unique.

Lambda functions that need to persist network connections, rely on ephemeral data created during initialization, or have strict requirements for randomness will require extra attention. The SnapStart documentation and AWS blog are helpful for understanding the code changes required to support these scenarios. For example, a common mistake is using a Pseudorandom Number Generator (PRNG) such as java.lang.Math.random() when a Cryptographically Secure Pseudorandom Number Generator (CPRNG) such as java.security.SecureRandom() will ensure your application code generates random values as expected with SnapStart.

Configuring Lambda Functions to work with SnapStart in Pulumi

To take advantage of faster cold starts, you must create your function resource with the option to use SnapStart. Note that beyond the previously described requirements to make your function code SnapSafe, no changes need to be made to the function code itself to use SnapStart. In the following example, we use the AWS Cloud Control provider to provision our function with SnapStart. The AWS Cloud Control provider uses the new Cloud Control API behind the scenes and has same-day support for new resources included in Cloud Control API - like SnapStart. This example also uses the AWS provider and the Pulumi Command Provider.

First, we add the necessary imports, create a role for our function to assume, and an S3 bucket to hold our Lambda code. Then we add our function’s zipped code as an object in the bucket. For this example, the Lambda function code is the AWS pet store example in Spring Boot:

import * as pulumi from "@pulumi/pulumi";
import * as awsNative from "@pulumi/aws-native";
import * as aws from "@pulumi/aws";
import * as command from "@pulumi/command";

const role = new aws.iam.Role("role", {
 assumeRolePolicy: JSON.stringify({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com",
 },
 "Action": "sts:AssumeRole",
 }],
 }),
});

new aws.iam.RolePolicyAttachment("role-policy-attachment", {
 role: role.name,
 policyArn: "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
});

const bucket = new aws.s3.Bucket("snapstart-bucket", {
 versioning: {
 enabled: true,
 }
});

const functionCode = new aws.s3.BucketObject("function-code", {
 bucket: bucket.bucket,
 source: new pulumi.asset.FileArchive("../petstore.zip"),
});

Next, we create our function with SnapStart enabled using the AWS Cloud Control provider, publish a version of our function (a requirement to use SnapStart) via the AWS CLI, and create a function alias for the version we created:

const func = new awsNative.lambda.Function("snapstart-func", {
 code: {
 s3Bucket: functionCode.bucket,
 s3Key: functionCode.key,
 s3ObjectVersion: functionCode.versionId,
 },
 role: role.arn,
 runtime: "java11",

 handler: "com.amazonaws.serverless.sample.springboot2.StreamLambdaHandler::handleRequest",
 memorySize: 1512,
 timeout: 60,
 snapStart: {
 applyOn: "PublishedVersions",
 },
});

const publishVersion = new command.local.Command("publish-version", {
 create: func.arn.apply(arn => `aws lambda publish-version --function-name ${arn}`),
 triggers: [func],
}, {
 dependsOn: func,
});

const aliasName = "v1";

const alias = new aws.lambda.Alias("alias", {
 functionName: func.arn,
 functionVersion: "1",
 name: aliasName,
}, {
 dependsOn: publishVersion,
});

Finally, we create the necessary resources to integrate our function with API Gateway and add our test URL as a stack output. This allows us to access it from outside of our Pulumi program:

const api = new aws.apigatewayv2.Api("snapstart-api", {
 protocolType: "HTTP",
});

const integration = new aws.apigatewayv2.Integration("lambdaIntegration", {
 apiId: api.id,
 integrationType: "AWS_PROXY",
 integrationUri: alias.arn,
 integrationMethod: "GET",
 payloadFormatVersion: "1.0",
 passthroughBehavior: "WHEN_NO_MATCH",
 connectionType: "INTERNET"
});

const route = new aws.apigatewayv2.Route("apiRoute", {
 apiId: api.id,
 routeKey: "$default",
 target: pulumi.interpolate`integrations/${integration.id}`,
});

new aws.apigatewayv2.Stage("apiStage", {
 apiId: api.id,
 name: "$default",
 routeSettings: [
 {
 routeKey: route.routeKey,
 throttlingBurstLimit: 5000,
 throttlingRateLimit: 10000,
 },
 ],
 autoDeploy: true,
}, { dependsOn: [route] });

new aws.lambda.Permission("permission", {
 action: "lambda:InvokeFunction",
 principal: "apigateway.amazonaws.com",
 function: alias.arn,
 sourceArn: api.executionArn.apply(x => `${x}/*/*`),
});

export const apiUrl = pulumi.concat(api.apiEndpoint, "/pets");

Deployment and Performance Testing

To deploy our function, we simply run the pulumi up command:

pulumi up -y

As the command completes, you’ll see output like the following:

Updating (dev)

View Live: https://app.pulumi.com/jkodroff/blog-lambda-snapshots-springboot-snapstart/dev/updates/9

 Type Name Status Info
 + pulumi:pulumi:Stack blog-lambda-snapshots-springboot-snapstart-dev created (4s) 1 warning
 + ├─ aws:iam:Role role created (0.82s)
 + ├─ aws:s3:Bucket snapstart-bucket created (2s)
 + ├─ aws:apigatewayv2:Api snapstart-api created (1s)
 + ├─ aws:iam:RolePolicyAttachment role-policy-attachment created (0.24s)
 + ├─ aws:s3:BucketObject function-code created (13s)
 + ├─ aws-native:lambda:Function snapstart-func created (22s)
 + ├─ command:local:Command publish-version created (3s)
 + ├─ aws:lambda:Alias alias created (0.66s)
 + ├─ aws:lambda:Permission permission created (0.40s)
 + ├─ aws:apigatewayv2:Integration lambdaIntegration created (0.74s)
 + ├─ aws:apigatewayv2:Route apiRoute created (0.54s)
 + └─ aws:apigatewayv2:Stage apiStage created (1s)


Outputs:
 apiUrl: "https://56r9255ey4.execute-api.us-west-2.amazonaws.com/pets"

Resources:
 + 13 created

Duration: 52s

We invoke our function by issuing an HTTP GET request against our stack output:

$ curl $(pulumi stack output apiUrl)
[{"id":"24ecd2bc-5106-46f8-95f2-af7de19cba57","breed":"Bloodhound","name":"Cooper","dateOfBirth":1639148877751},{"id":"caf6e8fa-fc97-4b9c-bfe7-996f14dc7995","breed":"Bloodhound","name":"Zoe","dateOfBirth":1438614477751},{"id":"b3d28309-8c92-4ae8-b7bd-7b656a5ea0bf","breed":"Beagle","name":"Murphy","dateOfBirth":1282489677751},{"id":"3c572757-298a-4343-9ca7-5c0865f9fce1","breed":"Beagle","name":"Winston","dateOfBirth":1410188877751},{"id":"4f8ab29a-760d-4b29-801d-6e35a98c538b","breed":"Dalmatian","name":"Max","dateOfBirth":1619968077751},{"id":"31fe978f-5eb6-4116-9933-ba8d9edc89a8","breed":"Dalmatian","name":"Sasha","dateOfBirth":1615475277751},{"id":"772218c3-cc56-44b7-87be-1db1b46730a3","breed":"Afghan Hound","name":"Lucky","dateOfBirth":1579187277751},{"id":"e994816e-8a6a-448a-bc48-ed404b463e46","breed":"Dalmatian","name":"Roxy","dateOfBirth":1408806477752},{"id":"3987cae6-6db2-48f9-96cb-dcb9812b1ab1","breed":"Bernese Mountain Dog","name":"Bailey","dateOfBirth":1329059277752},{"id":"514c8db6-225e-4fab-8c5b-63acdc88da56","breed":"Bloodhound","name":"Maggie","dateOfBirth":1218812877752}]%

To benchmark performance, use the Apache ab utility which is installed by default on macOS. The following command will run 1000 GET requests against our function:

ab -n 1000 -c 50 $(pulumi stack output apiUrl)

Comparing performance before and after SnapStart

During extensive benchmarking, the Pulumi team tried both a simple “Hello World” function as well as the more complex Spring Boot example above. Invoking this example Lambda function with and without SnapStart over many iterations, the Pulumi team found cold start times for a simple “Hello World” were 40% faster with SnapStart. The team found a much larger performance improvement when deploying the SpringBoot example. Cold starts were more than 10x faster. In real-world usage, this will translate into more responsive applications and services.

Conclusion

Addressing the challenges of cold starts for AWS Lambda functions is a major performance improvement that will enable Lambda functions to be useful in more scenarios that were previously the domain of dedicated VMs and container services. A 10x improvement in latency with common frameworks is a generational leap that we expect many teams to find valuable as they seek to modernize their infrastructure while keeping services responsive for end users.

Check out our GitHub repo with examples to try out Java-based functions with and without SnapStart enabled and see for yourself how these improvements can improve performance.

Happy coding!

Build Self-Service DevOps with AWS Proton, Pulumi & CodeBuild

Isaac Harris — Wed, 16 Nov 2022 08:36:28 -0800

Self-service infrastructure is the holy grail of DevOps. When platform engineering teams can empower application teams to provision their own infrastructure without needing to understand the details of configuring networking, storage, and compute resources, IT organizations can drastically increase their ability to deliver on organizational goals. The first step in this process is to codify infrastructure best practices using platforms like Pulumi and the next step is to make these best practices available in a workflow that fits into the application team’s software development tools and process.

Today, AWS announced support for AWS CodeBuild as a provisioner in AWS Proton. Using CodeBuild as a provisioner allows Proton to integrate with the most popular Infrastructure as Code platforms, including Pulumi. With Codebuild provisioning, Proton enables platform teams to define the steps to provision resources based on their own custom workflow and IaC tool of choice.

What are AWS Proton and AWS CodeBuild?

AWS Proton is a service that enables platform engineers to define, vend, and maintain self-service infrastructure templates for deploying cloud resources in an automated workflow. Development and operations teams can leverage these templates to deploy well-architected, modern infrastructure with confidence. Platform teams can create Proton templates for environments (which typically include shared resources like VPCs and container scheduler clusters) and application services which utilize the resources in the environment. Templates may be stored in git repositories for easy version control and automatic syncing with the Proton service.

AWS CodeBuild is a managed continuous integration (CI) service. Using CodeBuild, engineers can specify a Docker build container image, either pre-built or custom, and a list of commands to execute in the container to perform the build. AWS handles concerns like creating and destroying the container, logging, and metrics. When using CodeBuild as a provisioner for AWS Proton, teams specify the steps to both provision and tear down our infrastructure templates.

Using AWS Proton with CodeBuild and Pulumi

Since CodeBuild provisioning in Proton allows us to execute arbitrary commands, we can script our template builds using the Pulumi CLI. AWS has built a complete, end-to-end example that can be found at GitHub, and for this post we are going to focus on the specific steps required to use CodeBuild provisioning in AWS Proton with Pulumi. Note that in this example we are still using the Pulumi Service as our backend, so we can still enjoy all the benefits the service provides like state file management, secure secrets handling, a shared history of changes to our stacks, and advanced features available in Enterprise and Business Critical editions.

Define Your Proton Manifest File

When using Pulumi with CodeBuild provisioning in AWS Proton, our main interface between Proton and Pulumi occurs in the manifest file where we define the steps to provision and deprovision our infrastructure. Our manifest file also contains mappings for Pulumi configuration. We define our manifest file in YAML and include it in the root of our Pulumi program.

The following is an example of a manifest file that uses CodeBuild provisioning with Proton and Pulumi:

infrastructure:
 templates:
 - rendering_engine: codebuild
 settings:
 image: aws/codebuild/amazonlinux2-x86_64-standard:4.0
 runtimes:
 nodejs: 16
 env:
 variables:
 PULUMI_ORG: <PULUMI-ORG-HERE>
 PULUMI_NAMESPACE: proton-service
 secrets_manager:
 PULUMI_ACCESS_TOKEN: "proton/pulumi-access-token:Secret"
 provision:
 - cat proton-inputs.json
 - curl -fsSL https://get.pulumi.com | sh
 - export PATH=$PATH:$HOME/.pulumi/bin
 - npm install
 - pulumi stack select $PULUMI_ORG/$PULUMI_NAMESPACE/$STACK_NAME || pulumi stack init $PULUMI_ORG/$PULUMI_NAMESPACE/$STACK_NAME
 - pulumi config set aws:region $AWS_DEFAULT_REGION
 - pulumi config set org $PULUMI_ORG
 - pulumi up --yes --stack $PULUMI_ORG/$PULUMI_NAMESPACE/$STACK_NAME
 - chmod +x ./pulumi-to-proton-outputs.sh
 - pulumi stack output --json | ./pulumi-to-proton-outputs.sh > outputs.json
 - aws proton notify-resource-deployment-status-change --resource-arn $RESOURCE_ARN --status IN_PROGRESS --outputs file://./outputs.json
 deprovision:
 - curl -fsSL https://get.pulumi.com | sh
 - export PATH=$PATH:$HOME/.pulumi/bin
 - npm install
 - pulumi stack select $PULUMI_ORG/$PULUMI_NAMESPACE/$STACK_NAME
 - pulumi config set aws:region $AWS_DEFAULT_REGION
 - pulumi config set org $PULUMI_ORG
 - pulumi destroy --yes -s $PULUMI_ORG/$STACK_NAME

The provisioning steps in the manifest are as follows:

Download and install any necessary dependencies not already present in the CodeBuild container, like the Pulumi CLI.
Run the Pulumi commands to provision our infrastructure.
Map our Pulumi outputs to the necessary JSON format required by AWS Proton.

The deprovisioning steps are fairly similar to the provisioning steps:

Download and install any dependencies.
Run the Pulumi commands to tear down our infrastructure.

Create a Schema File for Self-Service Inputs

Proton allows application teams to provision infrastructure via a self-service UI. In order to inform the UI’s controls, platform engineering teams must supply Proton with a template schema file that defines the configurable parameters of the template.

The following is an example of a simple template schema file which allows us to configure the VPC CIDR block in our Proton environment:

schema:
 format:
 openapi: "3.0.0"
 environment_input_type: "EnvironmentInputs"
 types:
 EnvironmentInputs:
 type: object
 description: "Environment inputs"
 properties:
 vpcCidrBlock:
 title: "VPC CIDR block"
 type: string
 default: "10.0.0.0/16"

Package and Sync Templates

Finally, we need to package our template in a suitable format for Proton. Proton templates can be sourced from either S3 bucket objects or synced via git repositories. Using git as the source of truth for Proton templates is fully automated out of the box and thus is recommended over sourcing templates from S3 objects for production scenarios. For more information on git configuration with Proton, see Template sync configurations in the AWS docs.

A sample directory structure for an AWS Proton template that uses Pulumi follows:

.
├── infrastructure
│ ├── Pulumi.yaml
│ ├── index.ts
│ ├── manifest.yaml
│ ├── package-lock.json
│ ├── package.json
│ ├── proton-inputs.json
│ └── tsconfig.json
└── schema
└── schema.yaml

Our sample package contains 2 points of interest:

The infrastructure directory, which contains our Pulumi program plus the Proton manifest file.
The schema directory, which contains our template schema file.

For full details on Proton template packaging, see Wrap up template files for AWS Proton in the AWS docs.

Tying it all together

So what does this all look like from a user’s perspective when provisioning a self-service template? Let’s look at some screenshots.

First, the user is presented with a list of the templates published by the platform team:

After selecting the template to deploy and filling in any configurable input parameters, the user can view the CodeBuild logs and is presented with a link to the CodeBuild job:

Finally, the user can view the fully provisioned template and reference any generated outputs:

Conclusion

Using Proton with CodeBuild and Pulumi, we can enjoy all the benefits of self-service infrastructure along with the benefits of the Pulumi service like secrets management and stack history, while using real programming languages to allow us to ship infrastructure faster. Give it a try today!

Secure AWS Lambda with IAM ABAC Policies

Paul Stack — Tue, 19 Jul 2022 00:00:00 +0000

Event-driven, serverless functions have become a defining feature of many modern cloud architectures. With recent capabilities such as AWS Lambda URLs and AWS Lambda Containers, AWS has made it clear that Lambda Functions are a platform that teams can use to deliver increasingly sophisticated services without worrying about managing underlying compute resources.

Today, AWS announced another advancement for their Lambda Functions platform: Attribute-Based Access Control (ABAC). At its core, ABAC support brings more granular permissions that are automatically applied based on IAM role tags, Lambda tags, or both. This update builds on well-established Role-Based Access Control (RBAC) principles while making it possible to implement granular controls without permissions updates for every new user and resource.

What are Attributes?

Attributes are a key or key/value pair - in AWS, these attributes are called tags. Tags can be applied to multiple roles – for example identifying members of a team.

Across many teams, your organization may share a lot of common roles, and, in order to enforce the principle of least access, you may wish to restrict access across teams such that Developer Team members cannot access Ops Team resources and vice versa.

Using ABAC for AWS Lambda Functions

For this example, we’ll define a new role ‘abac-test’ as well as a team attribute that must have the value of either ‘developers’ or ‘ops’ to enable creating a Lambda function.

To get started, let’s first install the Pulumi AWS provider:

$ npm install @pulumi/aws

$ dotnet add package Pulumi.Aws

$ pip3 install pulumi_aws

$ go get github.com/pulumi/pulumi-aws/sdk/v5

Now, create a new IAM Role that a user can assume:

const role = new aws.iam.Role("abac-test", {
 assumeRolePolicy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Sid: "",
 Effect: "Allow",
 Principal: {
 AWS: "arn:aws:iam::MYACCOUNTID:root",
 },
 Action: "sts:AssumeRole",
 },
 ],
 }),
});

var role = new Aws.Iam.Role("abac-test", new Aws.Iam.RoleArgs
{
 AssumeRolePolicy = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "sts:AssumeRole" },
 { "Effect", "Allow" },
 { "Principal", new Dictionary<string, object?>
 {
 { "AWS", "arn:aws:iam::MYACCOUNTID:root" },
 } },
 },
 }
 },
 }),
});

role = aws.iam.Role("abac-test",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "AWS": "arn:aws:iam::MYACCOUNTID:root",
 },
 }],
 }),
 tags={
 "Team": "developers",
 })

role, err := iam.NewRole(ctx, "abac-test", &iam.RoleArgs{
 AssumeRolePolicy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Sid": "",
 "Effect": "Allow",
 "Principal": {
 "AWS": "arn:aws:iam::MYACCOUNTID:root"
 },
 "Action": "sts:AssumeRole"
 }]
 }`),
})
if err != nil {
 return err
}

 abacTest:
 type: aws:iam:Role
 properties:
 assumeRolePolicy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: sts:AssumeRole
 Effect: Allow
 Principal:
 AWS: arn:aws:iam::MYACCOUNTID:root

Let’s now create a new IAM Policy that allows a user to create Lambda functions:

const createPolicy = new aws.iam.Policy(
 "createLambda",
 {
 policy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Action: ["lambda:CreateFunction", "lambda:TagResource"],
 Resource: "arn:aws:lambda:*:*:function:*",
 Condition: {
 // the requesting resource must have a tag with
 // team = developers/ops
 StringEquals: {
 "aws:RequestTag/Team": ["developers", "ops"], // must match
 },
 // your request must contain these tags
 // eg. project, but you might not have this defined yet
 "ForAllValues:StringEquals": {
 "aws:TagKeys": "Team", // must exist
 },
 },
 },
 {
 // Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 Effect: "Allow",
 Action: [
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 ],
 Resource: "*",
 },
 {
 Effect: "Allow",
 // These are assume role policies
 Action: ["iam:ListRoles", "iam:PassRole"],
 Resource: "*",
 },
 ],
 }),
 },
);

var createPolicy = new Aws.Iam.Policy("createPolicy", new Aws.Iam.PolicyArgs
{
 PolicyDocument = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "lambda:CreateFunction",
 "lambda:TagResource",
 }
 },
 { "Resource", "arn:aws:lambda:*:*:function:*" },
 { "Condition", new Dictionary<string, object?>
 {
 // the requesting resource must have a tag with
 // team = developers/ops
 { "StringEquals", new Dictionary<string, object?>
 {
 { "aws:RequestTag/Team", new[]
 {
 "developers",
 "ops",
 }
 },
 } },
 // your request must contain these tags
 // eg. project, but you might not have this defined yet
 { "ForAllValues:StringEquals", new Dictionary<string, object?>
 {
 { "aws:TagKeys", new[]
 {
 "Team",
 }
 },
 } },
 } },
 },
 new Dictionary<string, object?>
 {
 // Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 }
 },
 { "Resource", "*" },
 },
 new Dictionary<string, object?>
 {
 // These are assume role policies
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "iam:ListRoles",
 "iam:PassRole",
 }
 },
 { "Resource", "*" },
 },
 }
 },
 }),
});

create_policy = aws.iam.Policy("createPolicy", policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Action": [
 "lambda:CreateFunction",
 "lambda:TagResource",
 ],
 "Resource": "arn:aws:lambda:*:*:function:*",
 "Condition": {
 #the requesting resource must have a tag with
 #team = developers/ops
 "StringEquals": {
 "aws:RequestTag/Team": [
 "developers",
 "ops",
 ],
 },
 # your request must contain these tags
 # eg. project, but you might not have this defined yet
 "ForAllValues:StringEquals": {
 "aws:TagKeys": ["Team"],
 },
 },
 },
 {
 # Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 "Effect": "Allow",
 "Action": [
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 ],
 "Resource": "*",
 },
 {
 # These are assume role policies
 "Effect": "Allow",
 "Action": [
 "iam:ListRoles",
 "iam:PassRole",
 ],
 "Resource": "*",
 },
 ],
}))

createPolicy, err := iam.NewRolePolicy(ctx, "createPolicy", &iam.RoleArgs{
 Policy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Sid": "",
 "Effect": "Allow",
 "Condition": map[string]interface{}{
 "StringEquals": map[string]interface{}{
 "aws:RequestTag/Team": []string{
 "developers",
 "ops",
 },
 },
 "ForAllValues:StringEquals": map[string]interface{}{
 "aws:TagKeys": []string{
 "Team",
 },
 },
 },
 "Resource": "arn:aws:lambda:*:*:function:*",
 "Action": []string{
 "lambda:CreateFunction",
 "lambda:TagResource",
 },
 },
 {
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 },
 },
 {
 {
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:CreateFunction",
 "lambda:TagResource",
 },
 }
 }]
 }`),
})
if err != nil {
 return err
}

 createPolicy:
 type: aws:iam:Policy
 properties:
 policy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Effect: Allow
 Action:
 - lambda:CreateFunction
 - lambda:TagResource
 Resource: 'arn:aws:lambda:*:*:function:*'
 Condition:
 StringEquals:
 aws:RequestTag/Team:
 - developers
 - ops
 ForAllValues:StringEquals:
 aws:TagKeys:
 - Team
 - Effect: Allow
 Action:
 - lambda:GetFunction
 - lambda:ListVersionsByFunction
 - lambda:GetFunctionCodeSigningConfig
 Resource: '*'
 - Effect: Allow
 Action:
 - iam:ListRoles
 - iam:PassRole
 Resource: '*'

Notice, as part of the policy condition, we ensure that the requesting resource must have a tag with team: developers or team: ops.

Let’s ensure that this policy is attached to the Role we initially created:

const rpa = new aws.iam.RolePolicyAttachment("rpa", {
 policyArn: createPolicy.arn,
 role: role,
})

var rpa = new Aws.Iam.RolePolicyAttachment("rpa", new Aws.Iam.RolePolicyAttachmentArgs
{
 PolicyArn = createPolicy.Arn,
 Role = role.Name,
});

rpa = aws.iam.RolePolicyAttachment("rpa",
 policy_arn=create_policy.arn,
 role=role.name)

rpa, err := iam.NewRolePolicyAttachment(ctx, "rpa", &iam.RolePolicyAttachmentArgs{
 PolicyArn: createPolicy.Arn,
 Role: role.Name,
})
if err != nil {
 return err
}

 rpa:
 type: aws:iam:RolePolicyAttachment
 properties:
 policyArn: ${createPolicy.arn}
 role: ${abacTest.name}

We can deploy these resources using the command pulumi update:

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/c6fb4580-2706-47ee-8a56-b8942631254e

 Type Name Plan
 + pulumi:pulumi:Stack aws-abac-test-abac-testing create
 + ├─ aws:iam:Policy createLambda create
 + ├─ aws:iam:Role abac-test create
 + └─ aws:iam:RolePolicyAttachment createPolicy create

Resources:
 + 4 to create

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/1

 Type Name Status
 + pulumi:pulumi:Stack aws-abac-test-abac-testing created
 + ├─ aws:iam:Role abac-test created
 + ├─ aws:iam:Policy createLambda created
 + └─ aws:iam:RolePolicyAttachment createPolicy created

Resources:
 + 4 created

Duration: 7s

For the purposes of testing, we are now going to create a specific AWS Provider resource that we can pass the IAM Role details to assume.

const abacUserProvider = new aws.Provider(
 "assumeRole",
 {
 // assume the previously created role with the ABAC configured policy into our provider
 assumeRole: {
 roleArn: role.arn,
 sessionName: "abacTesting",
 },
 region: "us-east-1",
 },
 {
 // we want to depend on the role policy attachment being in place before we create the provider
 dependsOn: [rpa],
 }
);

var abacUserProvider = new Aws.Provider("abacUserProvider", new Aws.ProviderArgs
{
 AssumeRole = new Aws.Config.Inputs.AssumeRoleArgs
 {
 RoleArn = role.Arn,
 SessionName = "abacTesting",
 },
 Region = "us-east-1",
}, new CustomResourceOptions
{
 DependsOn =
 {
 rpa,
 },
});

abac_user_provider = pulumi.providers.Aws("abacUserProvider",
 assume_role=aws.config.AssumeRoleArgs(
 role_arn=role.arn,
 session_name="abacTesting",
 ),
 region="us-east-1",
 opts=pulumi.ResourceOptions(depends_on=[rpa]))

abacUserProvider, err := providers.Newaws(ctx, "abacUserProvider", &providers.awsArgs{
 AssumeRole: config.AssumeRole{
 RoleArn: role.Arn,
 SessionName: "abacTesting",
 },
 Region: "us-east-1",
}, pulumi.DependsOn([]pulumi.Resource{
 rpa,
}))
if err != nil {
 return err
}

 abacUserProvider:
 type: pulumi:providers:aws
 properties:
 assumeRole:
 roleArn: ${role.arn}
 sessionName: "abacTesting"
 region: "us-east-1"
 options:
 dependsOn:
 - ${rpa}

Now we can create an AWS Lambda with Pulumi as follows:

const lambdaRole = new aws.iam.Role("iamRole", {
 assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({
 Service: "lambda.amazonaws.com",
 }),
});

const deleteLambdaPolicy = new aws.iam.Policy(
 "deleteLambdaPolicy",
 {
 policy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Action: ["lambda:DeleteFunction"],
 Resource: "*",
 },
 ],
 }),
 },
 { parent: lambdaRole }
);

new aws.iam.RolePolicyAttachment(
 `deletePolicy`,
 {
 policyArn: deletePolicy.arn,
 role: role,
 },
 { parent: deletePolicy })

const lambda = new aws.lambda.Function(
 "lambdaFunction",
 {
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./app"),
 }),
 runtime: "nodejs12.x",
 role: lambdaRole.arn,
 handler: "index.handler",
 },
 { provider: abacUserProvider }
);

var lambdaRole = new Aws.Iam.Role("lambdaRole", new Aws.Iam.RoleArgs
{
 AssumeRolePolicy = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "sts:AssumeRole" },
 { "Effect", "Allow" },
 { "Principal", new Dictionary<string, object?>
 {
 { "Service", "lambda.amazonaws.com" },
 } },
 },
 }
 },
 }),
});
var deletePolicy = new Aws.Iam.Policy("deletePolicy", new Aws.Iam.PolicyArgs
{
 PolicyDocument = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "lambda:DeleteFunction" },
 { "Effect", "Allow" },
 { "Resource", "*" },
 },
 }
 },
 }),
});
var deletePolicyAttachment = new Aws.Iam.RolePolicyAttachment("deletePolicyAttachment", new Aws.Iam.RolePolicyAttachmentArgs
{
 Role = role.Name,
 PolicyArn = deletePolicy.Arn,
});
var lambdaFunction = new Aws.Lambda.Function("lambdaFunction", new Aws.Lambda.FunctionArgs
{
 Code = new FileArchive("./app"),
 Role = lambdaRole.Arn,
 Handler = "index.handler",
 Runtime = "nodejs14.x",
}, new CustomResourceOptions
{
 Provider = abacUserProvider,
});

lambda_role = aws.iam.Role("lambdaRole", assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com",
 },
 }],
}))
delete_policy = aws.iam.Policy("deletePolicy", policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "lambda:DeleteFunction",
 "Effect": "Allow",
 "Resource": "*",
 }],
}))
delete_policy_attachment = aws.iam.RolePolicyAttachment("deletePolicyAttachment",
 role=role.name,
 policy_arn=delete_policy.arn)
lambda_function = aws.lambda_.Function("lambdaFunction",
 code=pulumi.FileArchive("./app"),
 role=lambda_role.arn,
 handler="index.handler",
 runtime="nodejs14.x",
 opts=pulumi.ResourceOptions(provider=abac_user_provider))

lambdaRole, err := iam.NewRole(ctx, "lambdaRole", &iam.RoleArgs{
 AssumeRolePolicy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com"
 },
 "Action": "sts:AssumeRole"
 }]
 }`),
})
if err != nil {
 return err
}

deletePolicy, err := iam.NewRolePolicy(ctx, "deletePoicy", &iam.RoleArgs{
 Policy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:DeleteFunction",
 },
 }]
 }`),
})
if err != nil {
 return err
}

_, err = iam.NewRolePolicyAttachment(ctx, "deletePolicyAttachment", &iam.RolePolicyAttachmentArgs{
 Role: role.Name,
 PolicyArn: deletePolicy.Arn,
})
if err != nil {
 return err
}

_, err = lambda.NewFunction(ctx, "lambdaFunction", &lambda.FunctionArgs{
 Code: pulumi.NewFileArchive("./app"),
 Role: lambdaRole.Arn,
 Handler: pulumi.String("index.handler"),
 Runtime: pulumi.String("nodejs14.x"),
}, pulumi.Provider(abacUserProvider))
if err != nil {
 return err
}

 lambdaRole:
 type: aws:iam:Role
 properties:
 assumeRolePolicy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: sts:AssumeRole
 Effect: Allow
 Principal:
 Service: lambda.amazonaws.com
 deletePolicy:
 type: aws:iam:Policy
 properties:
 policy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: lambda:DeleteFunction
 Effect: Allow
 Resource: '*'
 deletePolicyAttachment:
 type: aws:iam:RolePolicyAttachment
 properties:
 role: ${abacRole.name}
 policyArn: ${deletePolicy.arn}
 lambdaFunction:
 type: aws:lambda:Function
 properties:
 code:
 Fn::FileArchive: ./app
 role: ${lambdaRole.arn}
 handler: index.handler
 runtime: nodejs14.x

If we try and deploy the lambda, we will get the following using the role we are going to assume then we will get an error as the lambda function has no Team tag.

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/7fdc10b2-5e44-497e-a067-f2e8b257496c

 Type Name Plan
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + ├─ aws:iam:Role iamRole create
 ├─ aws:iam:Role abac-test
 + │ └─ aws:iam:Policy deletePolicy create
 + │ └─ aws:iam:RolePolicyAttachment deletePolicy create
 + ├─ pulumi:providers:aws assumeRole create
 + └─ aws:lambda:Function lambdaFunction create

Resources:
 + 5 to create
 4 unchanged

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/2

 Type Name Status Info
 pulumi:pulumi:Stack aws-abac-test-abac-testing **failed** 1 error
 + ├─ aws:iam:Role iamRole created
 ├─ aws:iam:Role abac-test
 + │ └─ aws:iam:Policy deletePolicy created
 + │ └─ aws:iam:RolePolicyAttachment deletePolicy created
 + ├─ pulumi:providers:aws assumeRole created
 + └─ aws:lambda:Function lambdaFunction **creating failed** 1 error

Diagnostics:
 pulumi:pulumi:Stack (aws-abac-test-abac-testing):
 error: update failed

 aws:lambda:Function (lambdaFunction):
 error: 1 error occurred:
 * error creating Lambda Function (1): AccessDeniedException:
 status code: 403, request id: 4a8676f2-2b68-4d42-ad4b-f0e10ca31afd

Resources:
 + 4 created
 4 unchanged

Duration: 6s

If we add the correct Team tag to the Lambda function then the deploy will proceed as expected:

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/0afa267f-5f9f-43f5-9e00-d3ee19dbf2b8

 Type Name Plan
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + └─ aws:lambda:Function lambdaFunction create

Resources:
 + 1 to create
 8 unchanged

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/3

 Type Name Status
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + └─ aws:lambda:Function lambdaFunction created

Resources:
 + 1 created
 8 unchanged

Duration: 9s

Try It Out!

Lambda ABAC capabilities will help you to simplify governance by helping you to standardize roles and maintain security boundaries between teams. A common use case is using tags to restrict/enable the invoke action for a function in addition to the create action demonstrated above. Give Lambda ABAC functionality a try and let us know what you think in Community Slack.

PulumiUP Workshop Series

Matty Stratton — Tue, 03 May 2022 08:46:08 -0500

At Pulumi, we’re incredibly fortunate to have over 70 integration partners in our ecosystem – helping shared end-users to build, deploy and manage practically any cloud service they can imagine. Our most popular content often includes workshops that show end-users how to use these powerful integrations. This year for PulumiUP, we’re excited to announce that we’ve teamed up with a number of partners to deliver a workshop track that provides hands-on labs and demonstrations for a variety of platforms and scenarios.

Introduction to Infrastructure as Code

Thursday, May 5th at 2:30 PM PDT

We’ll explain the basics of IaC by exploring how to use Pulumi to build, configure and deploy a real-life, modern application using Docker.

From Zero to Production in Kubernetes

Tuesday, May 10th at 9:00 AM PDT

Our friends from NGINX will show you how to leverage the power of Python with Pulumi to define and manage your Kubernetes deployments and build powerful abstractions that make getting to production easier than ever before. You’ll learn about the new open-source Modern Application Reference Architecture (MARA) that helps you quickly deploy an app platform that includes Amazon EKS, NGINX Kubernetes ingress controller, logstore, logagent, cert-manager, Prometheus, Grafana, and more.

Kubecrash

Tuesday, May 17th at 9:00 AM PDT

Join Pulumi and the maintainers of Linkerd, cert-manager, and other projects for a series of crash courses about Kubernetes and cloud-native, open source technologies.

Getting Started with Azure and Infrastructure as Code

Wednesday, June 1st at 9:00 AM PDT

In this workshop, you will use Pulumi’s Azure Native Provider, built directly from the Azure API, to provision infrastructure with Pulumi’s Typescript SDK. Join Microsoft Sr. Cloud Advocate, April Edwards, and Pulumi Developer Advocate Matty Stratton for this introduction to IaC.

End-To-End Infrastructure, Apps, and Auth with Auth0

Tuesday, June 7th at 4:00 PM PDT

In this live-coding session, Pulumi and Auth0 will walk through the process of building a three-tier web app: a single-page app built with React, a back end consisting of a REST API managed with Express and MongoDB. Finally, we’ll show you how to enable authentication to restrict access to your app.

AWS Immersion Day: Hands-On Infrastructure as Code

Tuesday, June 14th at 9:00 AM PDT

Pulumi and friends from VirtusLab and AWS have planned an interactive session to introduce IaC concepts using familiar programming languages to provision modern cloud infrastructure. This session will focus on new Pulumi capabilities.

GitOps with GitHub Actions and the Pulumi Operator

Tuesday, June 21st at 9:00 AM PDT

Combining GitHub Actions with the Pulumi Kubernetes Operator helps you to implement powerful GitOps workflows and automation for both your infrastructure and workloads. We’ve partnered with the GitHub team to deliver step-by-step labs to help you to get started with GitOps.

CI/CD Pipelines for Kubernetes Apps with Pulumi & Codefresh

Wednesday, June 22nd at 9:30 AM PDT

The Codefresh team will help you to learn the basics of CI/CD and how to declare cloud resources and set up a simple pipeline for Kubernetes deployments using your favorite programming languages with Pulumi and Codefresh.

Deploying Microservices with AWS Lambda and Pulumi

Tuesday, June 28, 12:00 PM PDT

In this workshop, we’ll examine how Pulumi can rapidly accelerate provisioning of cloud infrastructure. We’ll focus on AWS Lambda and build an example set of microservices utilizing AWS’s newest Lambda features. Matty Stratton (Pulumi) and Marina Novikova (AWS) will guide you through the process of provisioning a set of example Lambda resources in AWS.

Getting Started with Infrastructure as Code on Oracle Cloud

Thursday, July 14th at 9:00 AM PDT Register

Now that we have a brand new integration with Oracle Cloud Infrastructure (OCI) the team is preparing a new workshop to help you to get the most out of the new provider. Laura Santamaria (Pulumi) and Gaurav Jain (Oracle) will show you how to define, deploy and manage OCI resources using your favorite programming languages.

Getting started with Infrastructure as Code on DigitalOcean

Thursday, July 28th at 8:00 AM PDT Register

In this hands-on workshop, Matty Stratton (Pulumi) and Chris Sevilleaja (DigitalOcean) will show you how to stand up basic services using Infrastructure as Code with hands-on labs using JavaScript/TypeScript.

EKS Blueprints for Pulumi

Isaac Harris — Wed, 20 Apr 2022 18:00:00 +0000

With the launch of Amazon Elastic Kubernetes Service (EKS) in 2017, it is now easier than ever to build, secure, operate and maintain Kubernetes clusters in the cloud. Notably, EKS removed the need to manage and configure underlying compute resources and scaling for clusters. Further, EKS Anywhere brings many benefits to hybrid and on-premises deployments.

These developments have proved to be a huge leap forward in productivity for teams that manage cloud infrastructure, enabling them to focus their efforts on deploying applications to meet the needs of customers and stakeholders.

As companies have matured in their use of Kubernetes, the day 0 through day 2 activities surrounding deployments have started to look similar regardless of where the workload is running. What has emerged is something we call a Shared Services Platform (SSP) – an internal company service that allows application developers to self-service infrastructure environments. SSPs are highly useful for platform teams when they want to share common infrastructure patterns and standardize/automate the provisioning of resources for their development teams. This allows the platform team to be more responsive to development team requests for new environments.

Common activities for teams managing SSPs include:

Setting up and managing multiple environments like DEV, TEST, STAGE, and PROD
Simplifying platform administration with addons
Onboarding Internal Teams via GitOps

Pulumi makes all of these activities easy to automate with built-in secrets management, reusable components, support for end-to-end infrastructure testing, and centralized policy enforcement. We also see new opportunities for simplification in all these activities. We’re excited to be part of the Amazon EKS Blueprints program to help Pulumi users deliver their platforms and applications faster than ever.

With today’s launch, we’re announcing a preview of a few of these scenarios that highlight the power of multi-language components that work across JavaScript/TypeScript, Python, .NET, and Golang in Pulumi. The goal is to get community feedback on these concepts and build more valuable components that meet the production needs of our users.

Here are some examples of how to simplify Kubernetes deployments using EKS Blueprint patterns built as Pulumi multi-language components:

Setting up and managing multiple environments

Pulumi has a concept called Stacks that allows you to deploy the same Pulumi program multiple times, with minor variations to account for environmental differences. Leveraging this with our new Pulumi SSP SDK, you can deploy a cluster to multiple environments with very little code. Here’s an example with one of Pulumi’s supported languages, Python.

from pulumi_ssp import SharedServicesPlatform, ClusterArgsArgs

ssp = SharedServicesPlatform("python-platform", cluster_args=ClusterArgsArgs(kubernetes_version="1.21.0", region="us-east-1"))

That’s it! With a single line of code, you can deploy an EKS cluster in your region of choice and the version of Kubernetes you want. You’ll get the default EKS node group with this configuration, but what if you want more control? Let’s take a look at that with another supported language, TypeScript.

import * as ssp from "@pulumi/ssp";

const sharedServicesPlatform = new ssp.SharedServicesPlatform("my-platform", {
 clusterArgs: {
 kubernetesVersion: "1.21.0",
 region: "us-east-1",
 },
});

sharedServicesPlatform.addManagedNodeGroup({
 name: "my-managed-group",
 desiredSize: 1,
 minSize: 1,
 kubernetesVersion: "1.21.0",
 maxSize: 2,
 instanceTypes: ["t3.medium"],
});

Our multi-language SDK provides some convenience functions that allow you to add NodeGroups and ManagedNodeGroups with just a few lines of configuration.

Simplifying Platform Administration with Addons

Of course, a Kubernetes cluster with nothing on it isn’t going to give you the platform your organization needs to onboard teams and provide the level of redundancy and resiliency you need. So how can you add common Kubernetes deployments to handle observability, monitoring, and other “day 1 + day 2” tasks? Addons.

Our Pulumi SSP also provides a clusterAddon convenience function that allows you to deploy things with a single line.

A must-have for all Kubernetes clusters is the metrics-server and node-problem-detector, so let’s add these. We’ll stick with TypeScript for this one.

sharedServicesPlatform.clusterAddon({
 name: "metrics-server",
});

sharedServicesPlatform.clusterAddon({
 name: "node-problem-detector",
 version: "0.8.10"
});

Of course, you can easily leverage Pulumi’s integration with a variety of observability tools like Datadog and SumoLogic as well. We will be building more addons over the next few iterations, and we encourage you to reach out on our Community Slack and let us know what you’d like to see.

Onboarding an Internal Team via GitOps

Lastly, Pulumi SSP allows platform teams to onboard teams to the platform, also with a convenience function call.

This time we’ll go wild and show what this looks like with CSharp.

var ssp = new SharedServicesPlatform("csharp", new SharedServicesPlatformArgs
 {
 ClusterArgs = new Pulumi.Ssp.Inputs.ClusterArgsArgs
 {
 KubernetesVersion = "1.21.0",
 Region = "us-east-1",
 }
 });

 ssp.OnboardTeam(new SharedServicesPlatformOnboardTeamArgs
 {
 Name = "payments",
 Controller = "pulumi",
 Repository = "github.com/awesome-org/payments-team/infra"
 });

Onboarding a team to the platform is a wrapper for creating a namespace. Soon, this will add resource quotas and a GitOps delivery pipeline. We’ll initially support ArgoCD and the Pulumi Operator, but support for FluxCD will be added shortly after.

Conclusion

The EKS Blueprints program is helping to simplify the process of managing a shared services platform on Kubernetes by defining standard patterns that every engineering team will find useful. We’d love to get your feedback on this approach and areas where you want to see more high-level components that simplify everyday infrastructure tasks. Check out these code examples and give us your feedback in our EKS Blueprints repo.

AWS Enterprise Container Management with Pulumi

Isaac Harris — Thu, 14 Apr 2022 00:00:00 +0000

Managing containers and Kubernetes clusters are consistently popular topic areas on the Pulumi blog and in our docs. Our customers regularly cite that Pulumi simplifies container management scenarios, making it the primary reason for choosing Pulumi to define, deploy and manage all of their cloud resources. This includes teams that are just starting their cloud journey and spinning up their first project, as well as teams that want to modernize their apps and services with cloud-native architectures or even scale from one to many clouds.

We’re excited to be launch partners for the new Enterprise Container Management category of the AWS Container Competency program because it perfectly encapsulates Pulumi’s capabilities across the entire lifecycle of container-based architectures and applications. These scenarios are critical to the success of every cloud engineering and DevOps team and include provisioning, governance, security, and observability across Amazon Elastic Kubernetes Service (EKS) and Amazon Elastic Container Service (ECS). Here is a handy guide to some of the features that make Pulumi a great Enterprise Container Management Solution and how Pulumi makes each container scenario easier for your team.

Provisioning Container Infrastructure

Pulumi manages multi-region, multi-account, multi-cluster Kubernetes deployments with ease on any cloud. With Pulumi Crosswalk for Kubernetes we’ve provided a set of Day 0 and Day 1 playbooks that take the guesswork out of the provisioning process from creating a control plane to deploying your apps and updating workers.

Managing AWS Container Services

Pulumi gives you the flexibility to pick the container services that meet the needs of your workloads and the requirements of your organization. Choosing the right services can be a challenge, so we’ve assembled a short overview to help you get started managing AWS containers. Once you’ve chosen your scheduler, Pulumi has a host of examples to guide you through deploying Amazon ECS and Amazon EKS. There is also a Pulumi EKS component that provides multi-language convenience functions and boilerplate to simplify EKS deployments.

Support for the Entire Kubernetes API

Once you’ve provisioned a cluster, the Pulumi Kubernetes Provider enables you to provision any resource available in the Kubernetes API. This provides native support for features like Helm as well as the ability to configure Namespacing providing powerful multi-tenancy capabilities for your clusters.

Cluster Observability

Logging and monitoring are critical capabilities for keeping tabs on the health and security of your clusters. Fortunately, Pulumi has a host of integrations that simplify these scenarios including open-source tools like metrics-server and Prometheus as well as support for industry-leading platforms like Amazon CloudWatch and Datadog.

Authentication and Authorization

Adhering to the principle of least privilege for users and roles is an important step in securing your clusters and Pulumi has many capabilities to help you manage the complexities of authentication and authorization. For example, with Pulumi you can create and manage AWS IAM roles and you can integrate Pulumi Enterprise with your centralized identity and access management platform of choice via SAML 2.0 and OIDC.

Centralized Governance and Compliance Controls

Many customers are using Pulumi and Kubernetes to stand up shared services platforms (SSP) to empower their developers to self-service new infrastructure environments. To keep these environments compliant with internal policies, Pulumi Business Critical Edition includes CrossGuard policy-as-code capabilities built-in. This helps operators to ensure that configuration mistakes won’t reach production with policies that are enforced organization-wide.

Support for Hybrid Deployments

Many users need to manage container-based workloads on AWS as well as on-prem and on other clouds. Fortunately, Pulumi has built-in support for Amazon ECS Anywhere, so teams can use the familiar ECS control plane regardless of where their workloads need to run. Pulumi also supports the EKS Distro which brings the familiar managed Kubernetes capabilities of EKS to on-prem clusters.

Automated Deployment of Infrastructure and Applications

One of the benefits of Pulumi is that it enables infrastructure and application development to leverage the same tooling and processes familiar to software engineers. Adding infrastructure to your CI/CD workflow is easy with Pulumi because it supports a wide variety of test frameworks, simplifies the process of integration testing using ephemeral environments, and includes a CI/CD Integration Assistant to guide you through the process of connecting Pulumi to popular platforms such as GitHub Actions, AWS Code Services and many more.

Once you have your cluster provisioned, you can also deploy workloads into your cluster with GitOps style workflows using the Pulumi Kubernetes Operator.

Conclusion

This is just a sample of the many ways that Pulumi makes it easier than ever to manage containers and Kubernetes clusters in enterprise scenarios. Give it a try for yourself and let us know what you think in our Community Slack.

The Industry Comes Together for the Cloud Engineering Summit

Isaac Harris — Wed, 06 Oct 2021 06:00:00 -0700

When we launched the first-ever Cloud Engineering Summit last year, we were humbled by the generosity and support from partners and sponsors across the industry who helped to make the summit a success. For this year’s event, we’re equally thrilled to have an amazing group of sponsors participating in the summit and it’s worth a quick ‘guided tour’ of the teams that are helping us to pull off this amazing event.

Amazon Web Services

AWS was an inaugural sponsor for the Cloud Engineering Summit with a talk from Alex Head about cost optimization. The AWS team returns again this year with four sessions: Emily Freeman will talk about Rethinking the SDLC, Rahul Sharma will share his team’s experience building the brand new Cloud Control API in How AWS architects APIs for hyper-scale, and Jenna Pederson will show you how to Bring software development practices to your infrastructure. Finally, Ken Exner will be chatting with Joe Duffy about the future of tooling for modern cloud infrastructure.

We’ll cap off the summit in a free, post-event workshop with Marina Novikova and Laura Santamaria showing you how to get started on AWS.

Apple

New to the Cloud Engineering Summit this year, cloud services experts Fariba Khan and Stephen Van Gordon will share their experiences managing infrastructure at scale in Lifecycle of a Pulumi Program. They’ll talk about building a custom state backend, using CI/CD tooling, and the Pulumi Automation API to empower teams to provision secure-by-default configurations automatically.

Google Cloud

Last year we hosted a fantastic talk from Jason (Jay) Smith who showed attendees how to stand up a serverless platform. This year, we’re proud to include Google Cloud as a sponsor and feature a keynote from Kelsey Hightower with Joe Duffy.

After the summit, you can also join Kat Cosgrove for free hands-on training showing you how to get started with Google Cloud.

Microsoft

The 2020 Cloud Engineering Summit featured Amanda Silver in a Future of Cloud Engineering panel and Adora Nwodo showed everyone how to manage apps and infrastructure in a single CI/CD pipeline. Adora will return to the summit again this year to talk about Software engineering in infrastructure engineering.

For those looking to expand their cloud engineering skillset after the summit, Matt Stratton will host a workshop to show attendees how to get started with Azure.

NGINX

We’ve spent a lot of time with the NGINX team over the past year in their quest to make it easier than ever to get to production with Kubernetes. Join first-time summit speakers Damian Curry and Elijah Zupancic for a lightning talk followed by NGINX office hours to learn about Building a modern app reference architecture for Kubernetes.

Then, if you’re ready for some hands-on Kubernetes help, join our post-event workshop From Zero to Production in Kubernetes.

Bridgecrew

New to the summit this year, Bridgecrew’s Steve Giguere is joining the Build track to share Taking a Kubernetes Deployment from Default to Secure.

CircleCI

Long-time collaborator Angel Rivera from CircleCI joins the Deploy track to talk about Building Arm Compatible CI/CD Pipelines and show you how to take advantage of Arm architectures in both your pipelines and workloads.

GitHub

The GitHub team participated in three sessions last year including the Architectures and Platforms and Security and Policy panels. This year, the team returns with Rizel Bobb-Semple delivering a plenary session showing you how to get From Code to Cloud without learning new tools.

Honeycomb

If you missed Charity Major’s summit session last year about Testing In Production, stop reading and go watch it. Then, register for Liz Fong-Jones’s Build track session this year: What is OpenTelemetry.

NS1

Last year, Johnathan Sullivan joined the Culture, Process, and Tools panel, and we’re excited to welcome Chris Buijs to this year’s Deploy panel.

…and many more

Beyond the teams and sessions mentioned above, the Cloud Engineering Summit wouldn’t be possible without help and support from a long list of sponsors and collaborators including Codefresh, Split, and Uffizzi. Watch the replay to see what all the fuss is about.

Infrastructure CI/CD With Github Actions and Pulumi

Paul Stack — Mon, 14 Dec 2020 00:00:00 +0000

GitHub Actions help automate tasks within your software development life cycle. At Pulumi, we use GitHub Actions internally as part of every build/release cycle, and we run these tasks many, many times per day. This helps us to automate our CI/CD process and eliminate manual steps. Pulumi and many of our customers also deliver infrastructure resources as part of a CI/CD process: delivering infrastructure and applications in a single, integrated pipeline.

Better Automation with GitHub Actions and Pulumi

Today, we’re pleased to announce that Pulumi is available as a verified GitHub Action in the GitHub Marketplace. It’s now easier than ever to start automating your infrastructure deployments with Pulumi and GitHub Actions. Our new guide shows you how to get started with GitHub Actions and Pulumi. Here are just a few great examples of how to use the Pulumi GitHub Action that will simplify your cloud engineering tasks:

Master Builds

name: Pulumi
on:
 push:
 branches:
 - master
jobs:
 up:
 name: Update
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v1
 - uses: pulumi/actions@v1
 with:
 command: up
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

Pull Request Builds

name: Pulumi
on:
 - pull_request
jobs:
 preview:
 name: Preview
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v2
 - uses: pulumi/actions@v1
 with:
 command: preview
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
 GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 IS_PR_WORKFLOW: true
 COMMENT_ON_PR: 1

Announcing the New Pulumi CI/CD Integration Assistant

We often hear from developers who want to setup CI/CD automation for their stacks but don’t have the time to figure out how to configure a working pipeline. Today, we are happy to announce the launch of the CI/CD Integration Assistant to make setting up CI/CD automation more straightforward. You can easily get started using popular CI/CD services without the hassle of the initial setup.

The integration assistant guides you with instructions based on your preferred CI/CD service and provides the customized pipeline configuration for CI/CD services that you can add to your Version Control System (VCS) repository.

We are super excited to offer GitHub Actions support as part of the CI/CD Integration Assistant and other popular tools like Azure Pipelines and more. If you’d like to request support for more CI/CD services, please use this convenient template on the Pulumi GitHub repo.

Managing Your Infrastructure with a VCS

You may already be familiar with Git-based services such as GitHub, Bitbucket, and GitLab. Adding your code to a central VCS allows you to secure your codebase against failures and boost team collaboration. When you add an automated CI/CD pipeline, your team will deliver software safely and efficiently by ensuring that automated tests are passed prior to every deployment. This process is valuable for application code and infrastructure code.

The Assistant also goes beyond helping you get started with an automated pipeline. It works throughout the Pulumi Service providing hints for configuring a VCS for projects that are not already configured. It can also help identify updates to stacks that were not run in an automated pipeline (in the activity tab). These additions will help you quickly identify the stacks where best practices are not followed and take corrective actions. You can access the Assistant in the Integrations option of your stack’s settings page to help you through the whole process of configuring a VCS to setting up a working automated pipeline.

Learn more about the CI/CD Integration Assistant in our user guide.

We are eager to hear your feedback on how we can improve your experience with this feature. If you haven’t already signed up for our Community Slack, it’s quick and easy! You can join in on conversations you like and get help from other community members, as well as the Pulumi Team. That’s it for now! 👋