Pulumi Blog: Hcl

Pulumi for All Your IaC — Including Terraform and HCL

Joe Duffy — Thu, 18 Dec 2025 14:00:00 -0800

We work with thousands of customers who prefer Pulumi due to our modern approach to infrastructure that delivers faster time to market with built-in security and compliance. Yet we know many organizations have years of investments into tools like Terraform. At the same time, HashiCorp customers are increasingly telling us about their frustrations post-IBM acquisition: rate increases, loss of open source heritage, overnight rug-pull of CDKTF, … and the hits just keep on coming. Today, we’re excited to announce three new ways Pulumi is enabling customers of HashiCorp, an IBM Company, who want a better, open source friendly, modern solution for their IaC to choose Pulumi. First, Pulumi Cloud will support Terraform and OpenTofu, so you can continue using any Terraform or Pulumi CLI and language with the complete Pulumi Cloud product, including our infrastructure engineering AI agent, Neo. Second, Pulumi’s own open source IaC tool will support HCL natively as one of its many languages, alongside the industry’s best languages including Python, TypeScript, Go, C#, Java, and YAML. Pulumi is multi-language at its core and many organizations are diverse and polyglot—these new capabilities truly make Pulumi the most universal IaC platform with the broadest support. Third, we’re offering flexible financing to make it easy to depart HashiCorp for Pulumi.

The TL;DR

Pulumi Cloud now manages Terraform/OpenTofu with full visibility, governance, and agentic AI included. Pulumi IaC now speaks HCL alongside general purpose languages and YAML. And we’ll cover your costs until your HashiCorp contract ends.

Terraform/OpenTofu in Pulumi Cloud

Pulumi has always been two things: Pulumi IaC, the multi-language, open source infrastructure as code technology, and Pulumi Cloud, our commercial platform that includes infrastructure state management and visibility, our AI agent Neo, secrets management, policy as code, governance, and much more. You can think of it like Git (the tool) versus GitHub (the SaaS).

Historically, we required that you choose Pulumi IaC to benefit from Pulumi Cloud but over time we’ve been moving away from that, such as with our AI and governance features: you point us at any cloud accounts, and we’ll help you make sense of and tame them, regardless of how the accounts’ resources were deployed (Pulumi, CDK, Terraform, … even manual point and click). Today, we are taking that one step further and letting you run Terraform, OpenTofu, or Pulumi IaC as your tool of choice, while still benefitting from Pulumi Cloud’s full suite of capabilities.

The great thing about this is that even if you choose Terraform or OpenTofu IaC on the client, you will still benefit from all of the capabilities of our server, from AI to governance to visibility and more. Terraform statefiles and workspaces will show up effectively the same way Pulumi stacks do, and you’ll get full visibility of who is changing what and when including diffs and logs.

Why would we do such a thing? As we’ve worked with larger and larger companies in our journey to thousands of customers, we’ve seen that there’s significant Terraform out there in the world. Even if a team’s long-term objective is to migrate to 100% Pulumi – reaping the many benefits of modern IaC, like faster time to market by catering better to a polyglot world of developers, infrastructure experts, security engineers, and AI/ML teams – that transition doesn’t happen overnight. Many teams legitimately want a mix of IaC tools. Ensuring all infrastructure is fully automated, secured, and managed is a more righteous outcome to focus on rather than debating one’s choice of IaC tool or language. There are many paths you can take, and now Pulumi can be your one platform to stay on top of all of it and drive towards this outcome.

Support for Terraform/OpenTofu state is generally available today, and we’re working with customers directly to bring their existing state under Pulumi Cloud’s governance, visibility, and Neo integration.

HCL Language Support in Pulumi IaC

At Pulumi, we love our languages. We now support six – depending on how you count: Python, TypeScript, Go, any .NET language (like C#), and any JVM language (like Java itself), and even YAML. Having this broad array of languages is a massive unlock: you suddenly get access to the full ecosystem of tooling and expertise around these languages, including rich syntax (for loops, if statements, functions), IDEs, testing frameworks, true sharing and reuse, and ensuring that LLMs deeply understand your IaC. This choice of language is then married with the best of declarative IaC, so you still get the belts and suspenders safety of a desired state IaC tool.

But we work with customers all the time where some of the team is more comfortable with and/or genuinely prefers HCL. The HCL language was purpose-built for IaC through Terraform and, now, OpenTofu and is easy for simple use cases. We actually shipped YAML support two years ago because it’s an industry standard and we kept hearing about simpler use cases where you didn’t need a full blown language (especially e.g. when code-generating IaC or supporting simple developer self-service CI/CD pipelines where a handful of lines of YAML do the trick). But despite that, there’s a ton of muscle memory with HCL in the IaC community.

We are not dogmatic about languages, we love all of them. The L in HCL and YAML stands for “language” and we’ve always had a “come one, come all” mindset. As soon as we see enough market demand for a given language, we will add it. Well, that time has come for HCL.

The good news is that this is not a bolt on. Just like any of the other Pulumi languages, you have full access to the entirety of the Pulumi ecosystem, including thousands of providers. Thanks to our Terraform bridge, if there’s a Terraform provider out there, it just works. This is explicitly not meant to be a lift-and-shift migration option – we have many other techniques for that, including the new Terraform support in Pulumi Cloud – but is instead for teammates who are more comfortable with or prefer HCL over general-purpose languages, but still want to leverage the more modern Pulumi IaC engine with its advanced multi-language capabilities.

HCL support also integrates with Pulumi’s multi-language technology in a deep way, so that you can author modules in one language and consume them from another. This will let, for example, platform teams author complex components in, say, Go – with the rich facilities offered by the language – and then expose them to teammates who consume them in HCL (or vice versa!)

Similar to Terraform support in Pulumi Cloud, HCL is now generally available as a first-class Pulumi language, alongside Python, TypeScript, Go, C#, Java, and YAML.

Builds on Existing Coexist/Convert Capabilities

These two new technical capabilities augment an existing array of tools that make it easy for you to choose Pulumi as your IaC platform of choice, even in organizations with hybrid client-side IaC tools, and languages may be general-purpose, HCL, or a mixture thereof.

Pulumi IaC supports using any Terraform provider. Many of them are available pre-built in the Pulumi Registry, however, in the event one isn’t pre-published, you can generate one on demand using the “Any Terraform Provider”. Pulumi doesn’t use the Terraform engine, but rather it leverages the resource schemas and create, read, update, and delete methods in them.

We have numerous migration tools. The first is Neo, our infrastructure engineering agent, who has Terraform-specific skills to migrate code and state. Neo leverages a number of building blocks that you can use directly instead. That includes the pulumi convert --from terraform command which understands how to convert Terraform/OpenTofu HCL or CDKTF code into a Pulumi language of your choosing, preserving code structure including migrating modules to Pulumi components. We also support directly importing resources from your cloud accounts directly, either at the CLI or visually in Pulumi Cloud. Read more about migration here.

You can deploy Terraform modules straight off the shelf from your favorite Pulumi language. That works as well for any supported Pulumi language, including HCL, and it shows up as though the underlying module resources were managed natively in Pulumi. This ensures if your team has a collection of battle tested best practices encoded as Terraform modules, and either plan to keep some Terraform or even migrate eventually, you don’t need to migrate right away.

Finally, you can reference Terraform state outputs from Pulumi programs and configurations. Using this you could, for example, keep your network defined in Terraform and define a layer of higher level infrastructure that consumes VPC and subnet IDs from that lower-layer network workspace. This can be useful if there’s short- or long-term coexistence between Terraform and Pulumi IaC programs, such as during an ongoing migration effort or in a hybrid team.

Pulumi Covered Until Your HashiCorp, an IBM Company, Renewal

Even with the new technical capabilities above, we realize many customers already have HashiCorp contracts that may have them locked into a single vendor. The last thing we want is for you to have to pay for two IaC solutions for a period of time. Additionally, we want to ensure that the transition is as smooth as possible and your team is set up for long-term success.

To make it easier to say yes to Pulumi, we are offering three things:

Escape hatch for your current contract. We know paying for two IaC solutions at once is a non-starter, so we’re letting you apply credits purchased from HashiCorp towards your Pulumi usage until your next renewal, avoiding double pay.
Free IaC modernization workshop. Our professional services cloud architects will host a free IaC modernization workshop to review where you’re at with your IaC already and share best practices of how to adopt the Pulumi platform at scale we’ve learned from working with world-class organizations like BMW, NVIDIA, and Supabase. You will leave this session trained up and equipped to succeed with the next phase of your IaC journey.
Return on investment (ROI) calculation. We will show you how the move to Pulumi will not only be spend-neutral thanks to the escape hatch, but how much value and savings you should expect to see, given our experience helping innovators like Snowflake accelerate their time to market – going from code to cloud in weeks to hours.

These ensure there’s no financial penalty for switching, a very clear ROI, and no learning curve. We have always been proud to work with customers of all sizes in all industries, so these offers are available to you whether you’re a Global 2000, startup, or somewhere in between.

Why Snowflake, BMW, and Wiz Switched to Pulumi

We’ve worked with thousands of companies over 8+ years to build what we view as the most innovative infrastructure as code platform on the market. We’re biased, of course, but our customers range from innovative startups, to established public technology companies, to Global 2000, and everywhere in between, and they tell us this all of the time.

For example:

Snowflake radically improved time to market on their path to IPO. “When we demonstrated to people that what used to take a week and a half now, with Pulumi, took under a day, they were shocked.”
Lemonade switched from Terraform to Pulumi so they could embed business logic into infrastructure, share and reuse logic, and scale their lean ops team to support a much larger group of developers. “We’re not limited to one-size-fits-all configurations, but can actually implement environment-specific customizations for our infrastructure.”
BMW was able to establish a center of infrastructure excellence that they call CodeCraft, standardizing all infrastructure delivery, and scaling to support 10,000+ developers.
Supabase was able to scale to meet the heightened demands and pace of AI, saying that “the infrastructure team acts as groundkeepers of our Pulumi practices, not gatekeepers, but promoters for the entire org."
Wiz manages over 1 million resources, tens of thousands Kubernetes clusters, and hundreds of data centers, with over 100,000 daily updates.

Here’s why:

Language choice. Especially with the addition of Terraform/OpenTofu support, and HCL language support natively in Pulumi IaC, the Pulumi platform truly is the most universal IaC platform available. It democratizes access to infrastructure across the entire organization which often has very varied and diverse skillsets. By embracing general-purpose languages, it allows engineering teams to apply rigor to how they manage infrastructure, improving productivity – translating directly to time to market – robustness, and standards compliance.

AI native. We’ve been infusing AI into our platform for over three years now, culminating in the release of our infrastructure engineering agent, Neo. Neo is like Claude Code for your infrastructure and allows you to automate short- and long-horizon infrastructure tasks, like spinning up and scaling infrastructure, upgrading clusters, getting compliant, reducing cloud waste, and so much more. Neo works over any infrastructure no matter how it was provisioned.

Standard enterprise capabilities and controls. Regardless of language or IaC tool choice, you still get one central standardized set of capabilities and controls. This ensures you have one “mission control” from which to standardize, secure, and govern your entire cloud estate, regardless of what client-side tool choice (or even in the presence of click-ops).

Full visibility of what’s happening, when, where, and why. The Pulumi platform always gives you total visibility into your cloud estate so you can quickly understand what is going on. This helps you wrap your hands around your cloud and chart a course to 100% IaC, the table stakes nirvana many organizations are trying to get to, and the choice of language is one small part.

The final point isn’t even a technical one, but we hear it from our customers all the time:

Customer love as our #1 company value. Our first company value is “when the customer is successful, we are too” – and we live it and breathe it daily. I just visited a new customer to review how the project is going yesterday and their head of cloud infrastructure and EVP of product were in the room and both went out of their way to express gratitude for how our team really showed up to help get them on the right track. We don’t view customers as a transaction, we view it as a lifetime partnership, and treat our customers with the respect they deserve. We won’t sleep at night until you’re on the cloud path that you envisioned when selecting Pulumi.

To learn more about our product capabilities, visit these pages:

Get Started Today

If you’d like to get started with the new Terraform/OpenTofu and HCL capabilities, or take advantage of the financial flexibility options, please get in touch.

We will work with you closely on a three-step process to adopt Pulumi: First, the modernization workshop; then a rapid proof of value; and finally an adoption plan that avoids you double paying Pulumi and HashiCorp.

If you’d like to try Pulumi on your own immediately, you can sign up for Pulumi Cloud here, or go through our open source IaC getting started tutorial here.

We can’t wait to earn the right to work together.

Pulumi vs HCL: Understanding the Language Differences in Infrastructure as Code

Engin Diri — Tue, 09 Jul 2024 00:00:00 +0000

The Java Language Architect at Oracle, Brian Goetz, author of Java Concurrency in Practice, has commented how declarative languages can be a double-edged sword:

HashiCorp’s infrastructure as code solution, Terraform, uses a domain-specific language (DSL) to declare cloud resources. Pulumi’s infrastructure as code solution, on the other hand, lets you choose from any number of modern languages – C#, Java, JavaScript, Go, Python, or TypeScript – or the industry-standard markup language YAML, to declare cloud resources. Although both Terraform and Pulumi are declarative infrastructure as code engines at their core, this fundamentally different approach to expression languages has significant consequences.

Real-World Consequences

We’ve worked with 1,000s of companies and 100,000+ developers and infrastructure experts alike to adopt infrastructure as code using Pulumi and modern languages, and we see several significant recurring themes as they migrate away from Terraform and HCL.

Tapping Into Millions of Developers

Pulumi allows developers to use widely-adopted general-purpose languages for infrastructure as code. This approach taps into vast existing developer communities, with JavaScript boasting over 17 million developers, Python 15 million, and Go 3 million worldwide. By utilizing these languages, Pulumi enables teams to leverage their current programming expertise and access extensive resources, libraries, and tools. Meanwhile, HCL maintains a dedicated user base of several hundred thousand developers who appreciate its specialized syntax for infrastructure description.

Pulumi Leverages Rich Ecosystems of General-Purpose Languages

Pulumi harnesses the extensive ecosystems surrounding popular programming languages for infrastructure as code. While domain-specific languages like HCL have their strengths, Pulumi’s approach offers developers access to:

Comprehensive IDE support, including features like autocompletion, syntax highlighting, and real-time error detection, which may be more advanced than those available for specialized IaC languages
Wide array of established linting tools to enforce coding standards and catch potential issues early in the development process
Robust testing frameworks that allow for thorough verification and validation of infrastructure configurations, often more mature than those specific to IaC languages
Extensive libraries and packages that can be seamlessly integrated into infrastructure code, expanding possibilities beyond traditional IaC toolsets

This approach allows teams to utilize familiar tools and workflows, potentially increasing productivity and code quality. Developers can leverage their existing programming expertise and the vast resources available in these language communities when working on infrastructure projects, which may require less additional learning compared to adopting a specialized syntax

Pulumi Embraces Open-Source Languages and Principles

Languages like Python, JavaScript, and Go are developed and maintained by diverse global communities, ensuring transparency and collective innovation. These languages have remained open-source throughout their history, with their core components freely available for use, modification, and distribution.

In contrast, some domain-specific IaC languages have different ownership and licensing models. For example, HCL has always been under the control of a single company. Recently, there have been changes in its licensing, with parts of its execution engine moving to a “source available” license that is no longer considered fully open-source.

By using mainstream open-source languages, Pulumi offers teams the benefits of well-established ecosystems, community-driven development, and the assurance of continued open availability. This approach can provide additional confidence for organizations concerned about long-term access and evolution of their IaC tooling.

Here’s a list of languages Pulumi commonly uses, along with their licenses:

Language	License
C#	MIT (.NET Core)
Java	GNU v2.0 (OpenJDK)
JavaScript	MIT (ECMAScript), BSD-style 3-Clause (V8)
Go	BSD-style 3-Clause
Python	BSD-style Python Software Foundation
TypeScript	Apache 2.0
YAML	Creative Commons Attribution

Developers Onboard Quicker with Pulumi

While HCL serves its purpose in infrastructure as code, many developers find Pulumi’s approach more appealing. Pulumi allows engineers to use familiar programming languages, making it easier to leverage existing skills and tools.

This familiarity often leads to faster onboarding and more efficient debugging processes. Pulumi’s flexibility enables developers to create custom solutions with ease, tapping into the full power of general-purpose languages. Additionally,

Pulumi’s open architecture provides developers with the freedom to integrate various tools and services, fostering a diverse and adaptable ecosystem for infrastructure management.

Enhancing Team Collaboration With Pulumi

Pulumi offers several advantages for fostering better collaboration across teams compared to HCL. While HCL is a powerful tool primarily utilized by operations and DevOps teams, its specialized nature for infrastructure tasks might limit broader engagement with development teams who use other programming languages.

Pulumi, on the other hand, supports multiple languages, allowing development teams to use familiar tools and languages which can enhance collaboration and streamline processes. Additionally, Pulumi’s flexibility in integrating with a variety of tools can help bridge gaps between different teams, ensuring smoother communication and more effective teamwork.

This inclusive approach not only accelerates progress but also democratizes access to infrastructure management, avoiding bottlenecks and enhancing overall team efficiency.

Reducing Redundancy and Copy-Paste Challenges

HCL often encounters a significant challenge with “copy-pasta,” where developers frequently copy and paste code. This practice can lead to redundant configurations across environments, bloating the codebase and complicating management.

Additionally, this increases the risk of errors as small mistakes can propagate across multiple instances, and each one must be manually updated when changes are needed, heightening the chance for inconsistencies. As projects evolve, maintaining a codebase with extensive copy-pasta becomes burdensome, slowing development and elevating the likelihood of bugs.

In contrast, Pulumi reduces the need for such redundancies by enabling more dynamic code reuse and better resource management, streamlining updates and fostering more robust and error-resistant infrastructure development.

Reduce Need for Custom Tooling

As projects expand, managing HCL configurations becomes increasingly complex, often necessitating custom tools for efficient large-scale management. While HCL is adept at defining infrastructure as code, it lacks built-in features to orchestrate, validate, and deploy configurations across extensive networks smoothly. Teams usually need to develop bespoke tools or integrate external solutions like Terragrunt, Atlantis, and Sentinel to automate tasks, ensure consistency, and facilitate integration with other systems. This process can demand significant resources and expertise.

In contrast, Pulumi is designed to scale seamlessly with projects, reducing the need for external tools or extensive custom solutions. Pulumi provides robust capabilities for managing large-scale projects efficiently, streamlining the deployment process, and ensuring consistency across various environments without the overhead of additional tooling. This makes Pulumi a more straightforward and resource-efficient choice for teams looking to manage their infrastructure effectively as they grow.

Future-Proofing Your Infrastructure

HCL, while robust, lacks the future-proof qualities of Pulumi SDKs due to its fixed syntax and usage constraints. In contrast, Pulumi’s support for multiple programming languages like Python, TypeScript, and Go, coupled with the potential to adopt new languages over time, offers unparalleled flexibility and adaptability.

This allows developers to write infrastructure as code in familiar languages, facilitating easier integration with existing projects and workflows. As technology progresses, the capacity to embrace new programming languages and paradigms becomes essential. Pulumi’s adaptable model supports this continuous evolution, ensuring that infrastructure as code practices can keep pace with the dynamic nature of development environments and preferences, thereby offering a more sustainable and scalable solution for long-term projects.

Making Infrastructure Code Simpler

As HCL continues to evolve, its fixed syntax and structured approach are likely to lead to greater complexity. This evolution could make the language increasingly specialized and challenging for newcomers, particularly those already proficient in more widely-used programming languages. As HCL adapts more bespoke features to meet specific needs, it may become less intuitive and harder to handle effectively, potentially resulting in a language that is cumbersome to use and maintain, especially in long-term projects.

In contrast, Pulumi offers a more streamlined and adaptable alternative. Supporting multiple mainstream programming languages like Python, TypeScript, and Go, Pulumi enables developers to work with tools and languages they are already familiar with, greatly reducing the learning curve and enhancing productivity. This flexibility allows Pulumi to integrate seamlessly into existing projects and workflows, making it a more sustainable and scalable choice for modern infrastructure needs.

HCL in the Wild: Practical Comparisons With Pulumi

In this section, we’ll explore real-life examples to compare and contrast HCL with Pulumi programs. By examining how each approach handles similar infrastructure tasks, you’ll quickly appreciate the advantages of leveraging a familiar programming language through Pulumi.

This comparison aims to highlight the practical benefits and increased efficiency that Pulumi offers over traditional HCL configurations.

`for_each` Loop With Conditional Expression

First, let’s look at an HCL example featuring a for_each loop with a conditional statement. At first glance, it may seem straightforward, but we can immediately see the unique way a loop must be defined in HCL.

Initially, you must decide whether to use count or for_each. Following this, you must adhere to the strict format for implementing conditional statements.

This contrasts starkly with how you would use these constructs in a generic programming language.

variable "storage_account_names" {
 type = list(string)
 default = ["mickey", "donald", "goofy"]
}

resource "azurerm_resource_group" "example" {
 name = "storage-rg"
 location = "West Europe"
}

resource "azurerm_storage_account" "my_storage" {
 for_each = {
 for name in var.storage_account_names :
 name => name
 if name != "goofy"
 }
 name = each.value
 resource_group_name = azurerm_resource_group.example.name
 location = azurerm_resource_group.example.location
 account_tier = "Standard"
 account_replication_type = "GRS"
}

Now let’s look at how we can approach the same example using Pulumi.

const storageAccountNames: string[] = [
 "mickey",
 "donald",
 "goofy",
];
const example = new azure.core.ResourceGroup("example", {
 name: "storage-rg",
 location: "West Europe",
});

storageAccountNames.filter((name: string) => name !== "goofy").forEach((name: string) => {
 new azure.storage.Account(`my_storage-${name}`, {
 name: name,
 resourceGroupName: example.name,
 location: example.location,
 accountTier: "Standard",
 accountReplicationType: "GRS",
 });
});

import pulumi
import pulumi_azure as azure

storage_account_names = ["mickey", "donald", "goofy"]

example = azure.core.ResourceGroup(
 "example",
 name="storage-rg",
 location="West Europe"
)

for name in filter(lambda n: n != "goofy", storage_account_names):
 azure.storage.Account(
 f"my_storage-{name}",
 name=name,
 resource_group_name=example.name,
 location=example.location,
 account_tier="Standard",
 account_replication_type="GRS"
 )

package main

import (
 "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/core"
 "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/storage"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 storageAccountNames := []string{"mickey", "donald", "goofy"}

 example, err := core.NewResourceGroup(ctx, "example", &core.ResourceGroupArgs{
 Name: pulumi.String("storage-rg"),
 Location: pulumi.String("West Europe"),
 })
 if err != nil {
 return err
 }

 for _, name := range storageAccountNames {
 if name != "goofy" {
 _, err := storage.NewAccount(ctx, "my_storage-"+name, &storage.AccountArgs{
 Name: pulumi.String(name),
 ResourceGroupName: example.Name,
 Location: example.Location,
 AccountTier: pulumi.String("Standard"),
 AccountReplicationType: pulumi.String("GRS"),
 })
 if err != nil {
 return err
 }
 }
 }

 return nil
 })
}

using Pulumi;
using Pulumi.Azure.Core;
using Pulumi.Azure.Storage;
using System.Collections.Generic;

class Program
{
 static Task<int> Main() => Deployment.RunAsync<MyStack>();
}

class MyStack : Stack
{
 public MyStack()
 {
 var storageAccountNames = new List<string> { "mickey", "donald", "goofy" };

 var resourceGroup = new ResourceGroup("example", new ResourceGroupArgs
 {
 Name = "storage-rg",
 Location = "West Europe",
 });

 foreach (var name in storageAccountNames)
 {
 if (name != "goofy")
 {
 var storageAccount = new Account($"my_storage-{name}", new AccountArgs
 {
 Name = name,
 ResourceGroupName = resourceGroup.Name,
 Location = resourceGroup.Location,
 AccountTier = "Standard",
 AccountReplicationType = "GRS",
 });
 }
 }
 }
}

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.azurenative.core.ResourceGroup;
import com.pulumi.azurenative.core.ResourceGroupArgs;
import com.pulumi.azurenative.storage.Account;
import com.pulumi.azurenative.storage.AccountArgs;

import java.util.List;

public class Main {
 public static void main(String[] args) {
 Pulumi.run(Main::stack);
 }

 public static void stack(Context ctx) {
 var storageAccountNames = List.of("mickey", "donald", "goofy");

 var resourceGroup = new ResourceGroup("example", ResourceGroupArgs.builder()
 .name("storage-rg")
 .location("West Europe")
 .build());

 for (var name : storageAccountNames) {
 if (!name.equals("goofy")) {
 var storageAccount = new Account("my_storage-" + name, AccountArgs.builder()
 .name(name)
 .resourceGroupName(resourceGroup.name())
 .location(resourceGroup.location())
 .kind("StorageV2")
 .sku(AccountSkuArgs.builder()
 .name("Standard_GRS")
 .tier("Standard")
 .build())
 .build());
 }
 }
 }
}

We can see that we leverage the full potential that the generic programming language offers, using filter to streamline our array without the need for conditional statements, and then iterating through the reduced array with forEach. This approach is less error-prone, faster, and easier to debug.

However, implementing a for_each in HCL can become even more complicated.

`for_each` Loop With Multi-Level Nested Dynamic Blocks

To give you an idea of how HCL handles cascading nested dynamic providers, consider this: dynamic providers are intended to dynamically construct repeatable nested blocks. However, this complexity can make the code difficult to read and maintain. We will examine a specific resource from this GitHub repository to provide a clearer illustration:

variable "enable_auto_scaling" {
 type = bool
 default = false
 description = "Enable node pool autoscaling"
}

variable "agents_pool_kubelet_configs" {
 type = list(object({
 cpu_manager_policy = optional(string)
 cpu_cfs_quota_enabled = optional(bool, true)
 cpu_cfs_quota_period = optional(string)
 image_gc_high_threshold = optional(number)
 image_gc_low_threshold = optional(number)
 topology_manager_policy = optional(string)
 allowed_unsafe_sysctls = optional(set(string))
 container_log_max_size_mb = optional(number)
 container_log_max_line = optional(number)
 pod_max_pid = optional(number)
 }))
 default = [
 {
 cpu_manager_policy = "none"
 cpu_cfs_quota_enabled = true
 cpu_cfs_quota_period = "100ms"
 image_gc_high_threshold = 85
 image_gc_low_threshold = 65
 topology_manager_policy = "none"
 allowed_unsafe_sysctls = ["net.*", "kernel.*"]
 container_log_max_size_mb = 10
 container_log_max_line = 10
 pod_max_pid = 100
 }
 ]
 nullable = false
}

resource "azurerm_resource_group" "example" {
 name = "storage-rg"
 location = "West Europe"
}

resource "azurerm_kubernetes_cluster" "main" {
 name = "example-aks"
 location = azurerm_resource_group.example.location
 resource_group_name = azurerm_resource_group.example.name
 dns_prefix = "exampleaks"

 dynamic "default_node_pool" {
 for_each = var.enable_auto_scaling == true ? [] : ["default_node_pool_manually_scaled"]
 content {
 name = "default"
 vm_size = "Standard_DS2_v2"
 zones = [1, 2, 3]
 enable_auto_scaling = true
 min_count = 1
 max_count = 3
 dynamic "kubelet_config" {
 for_each = var.agents_pool_kubelet_configs
 content {
 allowed_unsafe_sysctls = kubelet_config.value.allowed_unsafe_sysctls
 }
 }
 }
 }
 identity {
 type = "SystemAssigned"
 }
}

As we can see, incorporating even one nested dynamic provider block can make the code difficult to read. We recommend using dynamic blocks sparingly to maintain readability and ease of maintenance, particularly when creating an abstraction with the help of a module. Overuse of these blocks, especially when defining a resource’s arguments and nested blocks directly from input variables, might not provide a useful abstraction.

Have look how this would look like when using Pulumi:

const config = new pulumi.Config();
const enableAutoScaling = config.getBoolean("enableAutoScaling") || false;

const agentsPoolKubeletConfigs = config.getObject("agentsPoolKubeletConfigs") || {
 allowedUnsafeSysctls: [
 "kernel.*",
 "net.*",
 ],
 containerLogMaxLine: 10,
 containerLogMaxSizeMb: 10,
 cpuCfsQuotaEnabled: true,
 cpuCfsQuotaPeriod: "100ms",
 cpuManagerPolicy: "none",
 imageGcHighThreshold: 85,
 imageGcLowThreshold: 65,
 podMaxPid: 100,
 topologyManagerPolicy: "none",
};
const example = new azure.core.ResourceGroup("example", {
 name: "storage-rg",
 location: "West Europe",
});

let kubeletConfigs: azure.types.input.containerservice.KubernetesClusterDefaultNodePoolKubeletConfig = agentsPoolKubeletConfigs;

let defaultNodePool: azure.types.input.containerservice.KubernetesClusterDefaultNodePool = {
 name: "default",
 vmSize: "Standard_DS2_v2",
 zones: [
 "1",
 "2",
 "3",
 ],
 enableAutoScaling: enableAutoScaling,
 minCount: 1,
 maxCount: 3,
 kubeletConfig: kubeletConfigs,
};

import pulumi
import pulumi_azure as azure

config = pulumi.Config()
enable_auto_scaling = config.get_bool("enableAutoScaling") or False

agents_pool_kubelet_configs = config.get_object("agentsPoolKubeletConfigs") or {
 "allowedUnsafeSysctls": [
 "kernel.*",
 "net.*",
 ],
 "containerLogMaxLine": 10,
 "containerLogMaxSizeMb": 10,
 "cpuCfsQuotaEnabled": True,
 "cpuCfsQuotaPeriod": "100ms",
 "cpuManagerPolicy": "none",
 "imageGcHighThreshold": 85,
 "imageGcLowThreshold": 65,
 "podMaxPid": 100,
 "topologyManagerPolicy": "none",
}

example = azure.core.ResourceGroup("example",
 name="storage-rg",
 location="West Europe")

kubelet_configs = azure.containerservice.KubernetesClusterDefaultNodePoolKubeletConfigArgs(
 **agents_pool_kubelet_configs
)

default_node_pool = azure.containerservice.KubernetesClusterDefaultNodePoolArgs(
 name="default",
 vm_size="Standard_DS2_v2",
 zones=["1", "2", "3"],
 enable_auto_scaling=enable_auto_scaling,
 min_count=1,
 max_count=3,
 kubelet_config=kubelet_configs,
)

package main

import (
 "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/core"
 "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/containerservice"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 enableAutoScaling := false

 agentsPoolKubeletConfigs := map[string]interface{}{
 "allowedUnsafeSysctls": []string{
 "kernel.*",
 "net.*",
 },
 "containerLogMaxLine": 10,
 "containerLogMaxSizeMb": 10,
 "cpuCfsQuotaEnabled": true,
 "cpuCfsQuotaPeriod": "100ms",
 "cpuManagerPolicy": "none",
 "imageGcHighThreshold": 85,
 "imageGcLowThreshold": 65,
 "podMaxPid": 100,
 "topologyManagerPolicy": "none",
 }

 example, err := core.NewResourceGroup(ctx, "example", &core.ResourceGroupArgs{
 Name: pulumi.String("storage-rg"),
 Location: pulumi.String("West Europe"),
 })
 if err != nil {
 return err
 }

 kubeletConfigs := &containerservice.KubernetesClusterDefaultNodePoolKubeletConfigArgs{
 AllowedUnsafeSysctls: pulumi.StringArray{
 pulumi.String("kernel.*"),
 pulumi.String("net.*"),
 },
 ContainerLogMaxLine: pulumi.Int(agentsPoolKubeletConfigs["containerLogMaxLine"].(int)),
 ContainerLogMaxSizeMb: pulumi.Int(agentsPoolKubeletConfigs["containerLogMaxSizeMb"].(int)),
 CpuCfsQuotaEnabled: pulumi.Bool(agentsPoolKubeletConfigs["cpuCfsQuotaEnabled"].(bool)),
 CpuCfsQuotaPeriod: pulumi.String(agentsPoolKubeletConfigs["cpuCfsQuotaPeriod"].(string)),
 CpuManagerPolicy: pulumi.String(agentsPoolKubeletConfigs["cpuManagerPolicy"].(string)),
 ImageGcHighThreshold: pulumi.Int(agentsPoolKubeletConfigs["imageGcHighThreshold"].(int)),
 ImageGcLowThreshold: pulumi.Int(agentsPoolKubeletConfigs["imageGcLowThreshold"].(int)),
 PodMaxPid: pulumi.Int(agentsPoolKubeletConfigs["podMaxPid"].(int)),
 TopologyManagerPolicy: pulumi.String(agentsPoolKubeletConfigs["topologyManagerPolicy"].(string)),
 }

 _, err = containerservice.NewKubernetesClusterDefaultNodePool(ctx, "defaultNodePool", &containerservice.KubernetesClusterDefaultNodePoolArgs{
 Name: pulumi.String("default"),
 VmSize: pulumi.String("Standard_DS2_v2"),
 Zones: pulumi.StringArray{pulumi.String("1"), pulumi.String("2"), pulumi.String("3")},
 EnableAutoScaling: pulumi.Bool(enableAutoScaling),
 MinCount: pulumi.Int(1),
 MaxCount: pulumi.Int(3),
 KubeletConfig: kubeletConfigs,
 }, pulumi.Parent(example))
 if err != nil {
 return err
 }

 return nil
 })
}

using Pulumi;
using Pulumi.AzureNative.Resources;
using Pulumi.AzureNative.ContainerService;
using System.Collections.Generic;

class MyStack : Stack
{
 public MyStack()
 {
 var enableAutoScaling = false;

 var agentsPoolKubeletConfigs = new Dictionary<string, object>
 {
 { "allowedUnsafeSysctls", new string[] { "kernel.*", "net.*" } },
 { "containerLogMaxLine", 10 },
 { "containerLogMaxSizeMb", 10 },
 { "cpuCfsQuotaEnabled", true },
 { "cpuCfsQuotaPeriod", "100ms" },
 { "cpuManagerPolicy", "none" },
 { "imageGcHighThreshold", 85 },
 { "imageGcLowThreshold", 65 },
 { "podMaxPid", 100 },
 { "topologyManagerPolicy", "none" }
 };

 var resourceGroup = new ResourceGroup("example", new ResourceGroupArgs
 {
 Name = "storage-rg",
 Location = "West Europe"
 });

 var kubeletConfigArgs = new KubernetesClusterDefaultNodePoolKubeletConfigArgs
 {
 AllowedUnsafeSysctls = agentsPoolKubeletConfigs["allowedUnsafeSysctls"] as string[],
 ContainerLogMaxLine = agentsPoolKubeletConfigs["containerLogMaxLine"] as int?,
 ContainerLogMaxSizeMb = agentsPoolKubeletConfigs["containerLogMaxSizeMb"] as int?,
 CpuCfsQuotaEnabled = agentsPoolKubeletConfigs["cpuCfsQuotaEnabled"] as bool?,
 CpuCfsQuotaPeriod = agentsPoolKubeletConfigs["cpuCfsQuotaPeriod"] as string,
 CpuManagerPolicy = agentsPoolKubeletConfigs["cpuManagerPolicy"] as string,
 ImageGcHighThreshold = agentsPoolKubeletConfigs["imageGcHighThreshold"] as int?,
 ImageGcLowThreshold = agentsPoolKubeletConfigs["imageGcLowThreshold"] as int?,
 PodMaxPid = agentsPoolKubeletConfigs["podMaxPid"] as int?,
 TopologyManagerPolicy = agentsPoolKubeletConfigs["topologyManagerPolicy"] as string
 };

 var defaultNodePool = new KubernetesClusterDefaultNodePool("defaultNodePool", new KubernetesClusterDefaultNodePoolArgs
 {
 Name = "default",
 VmSize = "Standard_DS2_v2",
 Zones = new[] { "1", "2", "3" },
 EnableAutoScaling = enableAutoScaling,
 MinCount = 1,
 MaxCount = 3,
 KubeletConfig = kubeletConfigArgs
 }, new CustomResourceOptions { Parent = resourceGroup });
 }
}

class Program
{
 static Task<int> Main() => Deployment.RunAsync<MyStack>();
}

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.azure.core.ResourceGroup;
import com.pulumi.azure.core.inputs.ResourceGroupArgs;
import com.pulumi.azure.containerservice.KubernetesClusterDefaultNodePool;
import com.pulumi.azure.containerservice.inputs.KubernetesClusterDefaultNodePoolArgs;
import com.pulumi.azure.containerservice.inputs.KubernetesClusterDefaultNodePoolKubeletConfigArgs;

import java.util.Map;
import java.util.HashMap;

public class MyStack {
 public static void main(String[] args) {
 Pulumi.run(ctx -> {
 boolean enableAutoScaling = false;

 Map<String, Object> agentsPoolKubeletConfigs = new HashMap<>();
 agentsPoolKubeletConfigs.put("allowedUnsafeSysctls", new String[]{"kernel.*", "net.*"});
 agentsPoolKubeletConfigs.put("containerLogMaxLine", 10);
 agentsPoolKubeletConfigs.put("containerLogMaxSizeMb", 10);
 agentsPoolKubeletConfigs.put("cpuCfsQuotaEnabled", true);
 agentsPoolKubeletConfigs.put("cpuCfsQuotaPeriod", "100ms");
 agentsPoolKubeletConfigs.put("cpuManagerPolicy", "none");
 agentsPoolKubeletConfigs.put("imageGcHighThreshold", 85);
 agentsPoolKubeletConfigs.put("imageGcLowThreshold", 65);
 agentsPoolKubeletConfigs.put("podMaxPid", 100);
 agentsPoolKubeletConfigs.put("topologyManagerPolicy", "none");

 var resourceGroup = new ResourceGroup("example", ResourceGroupArgs.builder()
 .name("storage-rg")
 .location("West Europe")
 .build());

 var kubeletConfigArgs = KubernetesClusterDefaultNodePoolKubeletConfigArgs.builder()
 .allowedUnsafeSysctls((String[]) agentsPoolKubeletConfigs.get("allowedUnsafeSysctls"))
 .containerLogMaxLine((Integer) agentsPoolKubeletConfigs.get("containerLogMaxLine"))
 .containerLogMaxSizeMb((Integer) agentsPoolKubeletConfigs.get("containerLogMaxSizeMb"))
 .cpuCfsQuotaEnabled((Boolean) agentsPoolKubeletConfigs.get("cpuCfsQuotaEnabled"))
 .cpuCfsQuotaPeriod((String) agentsPoolKubeletConfigs.get("cpuCfsQuotaPeriod"))
 .cpuManagerPolicy((String) agentsPoolKubeletConfigs.get("cpuManagerPolicy"))
 .imageGcHighThreshold((Integer) agentsPoolKubeletConfigs.get("imageGcHighThreshold"))
 .imageGcLowThreshold((Integer) agentsPoolKubeletConfigs.get("imageGcLowThreshold"))
 .podMaxPid((Integer) agentsPoolKubeletConfigs.get("podMaxPid"))
 .topologyManagerPolicy((String) agentsPoolKubeletConfigs.get("topologyManagerPolicy"))
 .build();

 var defaultNodePool = new KubernetesClusterDefaultNodePool("defaultNodePool", KubernetesClusterDefaultNodePoolArgs.builder()
 .name("default")
 .vmSize("Standard_DS2_v2")
 .zones(new String[]{"1", "2", "3"})
 .enableAutoScaling(enableAutoScaling)
 .minCount(1)
 .maxCount(3)
 .kubeletConfig(kubeletConfigArgs)
 .build(), new CustomResourceOptions().parent(resourceGroup));
 });
 }
}

In Pulumi, using TypeScript as an example, we can easily create new objects of a specific type and populate them with values from our configuration. If we need to add conditions, we can include our usual if statements. This approach greatly simplifies the code, making it easy to read and maintain for any future changes.

Execute Programming Code With `null_resource`, `local_file` and `provisioners`

There are sometimes situations where there is a strong need to execute a script written in a programming language. Most prominent use case is packaging up the source code for an AWS lambda function. Let’s have a look into an example from the Terraform AWS Lambda module

locals {
 python = (substr(pathexpand("~"), 0, 1) == "/") ? "python3" : "python.exe"
}

# Generates a filename for the zip archive based on the content of the files
# in source_path. The filename will change when the source code changes.
data "external" "archive_prepare" {
 count = var.create && var.create_package ? 1 : 0
 program = [local.python, "${path.module}/package.py", "prepare"]
 query = {
 paths = jsonencode({
 module = path.module
 root = path.root
 cwd = path.cwd
 })
 docker = var.build_in_docker ? jsonencode({
 docker_pip_cache = var.docker_pip_cache
 docker_build_root = var.docker_build_root
 docker_file = var.docker_file
 docker_image = var.docker_image
 with_ssh_agent = var.docker_with_ssh_agent
 docker_additional_options = var.docker_additional_options
 docker_entrypoint = var.docker_entrypoint
 }) : null
 artifacts_dir = var.artifacts_dir
 runtime = var.runtime
 source_path = jsonencode(var.source_path)
 hash_extra = var.hash_extra
 hash_extra_paths = jsonencode(
 [
 # Temporary fix when building from multiple locations
 # We should take into account content of package.py when counting hash
 # Related issue: https://github.com/terraform-aws-modules/terraform-aws-lambda/issues/63
 # "${path.module}/package.py"
 ]
 )
 recreate_missing_package = var.recreate_missing_package
 }
}

# This transitive resource used as a bridge between a state stored
# in a Terraform plan and a call of a build command on the apply stage
# to transfer a noticeable amount of data
resource "local_file" "archive_plan" {
 count = var.create && var.create_package ? 1 : 0
 content = data.external.archive_prepare[0].result.build_plan
 filename = data.external.archive_prepare[0].result.build_plan_filename
 directory_permission = "0755"
 file_permission = "0644"
}

# Build the zip archive whenever the filename changes.
resource "null_resource" "archive" {
 count = var.create && var.create_package ? 1 : 0
 triggers = {
 filename = data.external.archive_prepare[0].result.filename
 timestamp = var.trigger_on_package_timestamp ? data.external.archive_prepare[0].result.timestamp : null
 }
 provisioner "local-exec" {
 interpreter = [
 local.python, "${path.module}/package.py", "build",
 "--timestamp", data.external.archive_prepare[0].result.timestamp
 ]
 command = data.external.archive_prepare[0].result.build_plan_filename
 }
 depends_on = [local_file.archive_plan]
}

We see a confusing mix of different providers (local and null_resource) and even the use of provisioner here. This setup is overly complex to read and maintain, leading to a split-brain situation where some logic is in the code and the rest in HCL. It also becomes very challenging to debug, especially if the team lacks experience with the Python code.

In Pulumi, simplicity is key: simply write your Python code and seamlessly integrate it with Pulumi SDK calls. No additional steps are necessary, and there’s no need to run the Python binary separately to execute the file. It’s all about streamlining the process to make your life easier. For instance, consider this example where we read the name of an Azure Resource Group from a JSON file (config.json, illustrating how effortlessly Python can blend with Pulumi’s infrastructure management capabilities.

import json
import pulumi
import pulumi_azure_native as azure_native

# Load the JSON file
with open('config.json') as json_file:
 config = json.load(json_file)

# Extract values from the JSON file
resource_group_name = config['resourceGroupName']
location = config['location']

# Create an Azure Resource Group
resource_group = azure_native.resources.ResourceGroup(
 "resourceGroup",
 resource_group_name=resource_group_name,
 location=location
)

# Export the name of the resource group
pulumi.export('resourceGroupName', resource_group.name)

If Statements Constructed With `count`

variable "mode" {
 default = "not-spoke"
}

variable "restricted_net_hub_project_id" {
 default = "xxx"
}

data "google_compute_network" "vpc_restricted_net_hub" {
 count = var.mode == "spoke" ? 1 : 0
 name = "vpc-c-shared-restricted-hub"
 project = var.restricted_net_hub_project_id
}

In this HCL example, we see the count parameter being used in a way that stretches its original intent—essentially using it to conditionally create or skip resources based on the value of var.mode. This method, while functional, can make the code more difficult to understand and maintain, as it repurposes Terraform constructs in unintended ways.

Contrast this with Pulumi, which allows you to use native if statements from the programming language, such as TypeScript or Python. This native approach enhances readability and maintainability because it aligns with standard programming practices, making it easier for developers to follow and debug. Pulumi’s use of familiar programming constructs means you can write more intuitive and straightforward infrastructure code without bending the tool to fit unusual patterns.

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

const config = new pulumi.Config();
const mode = config.get("mode") || "";
const restrictedNetHubProjectId = config.get("restrictedNetHubProjectId") || "";
if (mode == "spoke") {
 const vpcSharedSpoke = gcp.compute.getNetworkOutput({
 name: "vpc-c-shared-spoke",
 project: restrictedNetHubProjectId,
 });
}

import pulumi
import pulumi_gcp as gcp

config = pulumi.Config()
mode = config.get("mode") or ""
restricted_net_hub_project_id = config.get("restrictedNetHubProjectId") or ""

if mode == "spoke":
 vpc_shared_spoke = gcp.compute.get_network_output(
 name="vpc-c-shared-spoke",
 project=restricted_net_hub_project_id,
 )

package main

import (
 "github.com/pulumi/pulumi-gcp/sdk/v7/go/gcp/compute"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 mode := ""

 if mode == "spoke" {
 _, err := compute.LookupNetwork(ctx, &compute.LookupNetworkArgs{
 Name: "vpc-c-shared-spoke",
 Project: "your-restrictedNetHubProjectId",
 })

 if err != nil {
 return err
 }
 }

 return nil
 })
}

using Pulumi;
using Pulumi.Gcp.Compute;

class Program
{
 static Task<int> Main() => Deployment.RunAsync<ComputeStack>();

 class ComputeStack : Stack
 {
 public ComputeStack()
 {
 var mode = "";

 if (mode == "spoke")
 {
 var network = Output.Tuple(Output.Create("vpc-c-shared-spoke"), Output.Create("your-restrictedNetHubProjectId"))
 .Apply(tuple => GetNetwork.InvokeAsync(new GetNetworkArgs
 {
 Name = tuple.Item1,
 Project = tuple.Item2
 }));

 // Handle potential errors if necessary
 network.Apply(n =>
 {
 if (n == null)
 {
 throw new Exception("Network not found");
 }
 });
 }
 }
 }
}

import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.gcp.compute.ComputeFunctions;
import com.pulumi.gcp.compute.inputs.GetNetworkArgs;

public class Main {
 public static void main(String[] args) {
 Pulumi.run(ctx -> {
 String mode = "";

 if ("spoke".equals(mode)) {
 Output<Tuple<String, String>> networkData = Output.tuple(Output.of("vpc-c-shared-spoke"), Output.of("your-restrictedNetHubProjectId"));

 Output<ComputedNetwork> network = networkData.applyValue(tuple ->
 ComputeFunctions.getNetwork(GetNetworkArgs.builder()
 .name(tuple.t1)
 .project(tuple.t2)
 .build()
 )
 );

 network.applyValue(n -> {
 if (n == null) {
 throw new RuntimeException("Network not found");
 }
 return null;
 });
 }
 });
 }
}

HCL Function Frenzy

Another source of extreme confusion and constant re-learning is the heavy usage of built-in functions within HCL expressions. At first glance, while it seems helpful for performing complex operations quickly, it can lead to overly complex expressions.

Here is an example of very complex expressions involving extensive function usage from the Google Network module:

locals {
 local_network_name = element(reverse(split("/", var.local_network)), 0)
 peer_network_name = element(reverse(split("/", var.peer_network)), 0)

 local_network_peering = "${var.prefix}-${local.local_network_name}-${local.peer_network_name}"
 local_network_peering_name = length(local.local_network_peering) < 63 ? local.local_network_peering :
 "${substr(local.local_network_peering, 0, min(58, length(local.local_network_peering)))}-${random_string.network_peering_suffix.result}"
 peer_network_peering = "${var.prefix}-${local.peer_network_name}-${local.local_network_name}"
 peer_network_peering_name = length(local.peer_network_peering) < 63 ? local.peer_network_peering :
 "${substr(local.peer_network_peering, 0, min(58, length(local.peer_network_peering)))}-${random_string.network_peering_suffix.result}"
}

In Pulumi, we can use the same functions from the programming language we pick for our infrastructure. This makes things easier because we don’t need to learn new, special functions like in HCL. We can also use extra libraries if we need them. Compared to HCL, where you have to learn unique functions, Pulumi lets you work with tools you might already know, making the whole process simpler and more flexible.

import * as pulumi from "@pulumi/pulumi";

const config = new pulumi.Config();
const localNetwork = config.require("localNetwork");
const peerNetwork = config.require("peerNetwork");
const prefix = config.require("prefix");

const localNetworkName = localNetwork.split("/").reverse()[0];
const peerNetworkName = peerNetwork.split("/").reverse()[0];

const localNetworkPeering = `${prefix}-${localNetworkName}-${peerNetworkName}`;
const peerNetworkPeering = `${prefix}-${peerNetworkName}-${localNetworkName}`;
const networkPeeringSuffix = "peering";
const localNetworkPeeringName = localNetworkPeering.length < 63
 ? localNetworkPeering
 : `${localNetworkPeering.substring(0, Math.min(58, localNetworkPeering.length))}-${networkPeeringSuffix}`;

const peerNetworkPeeringName = peerNetworkPeering.length < 63
 ? peerNetworkPeering
 : `${peerNetworkPeering.substring(0, Math.min(58, peerNetworkPeering.length))}-${networkPeeringSuffix}`;

export const outputs = {
 localNetworkName,
 peerNetworkName,
 localNetworkPeeringName,
 peerNetworkPeeringName,
};

import pulumi

config = pulumi.Config()
local_network = config.require("localNetwork")
peer_network = config.require("peerNetwork")
prefix = config.require("prefix")

local_network_name = local_network.split("/")[::-1][0]
peer_network_name = peer_network.split("/")[::-1][0]

local_network_peering = f"{prefix}-{local_network_name}-{peer_network_name}"
peer_network_peering = f"{prefix}-{peer_network_name}-{local_network_name}"
network_peering_suffix = "peering"

local_network_peering_name = (
 local_network_peering if len(local_network_peering) < 63
 else f"{local_network_peering[:min(58, len(local_network_peering))]}-{network_peering_suffix}"
)

peer_network_peering_name = (
 peer_network_peering if len(peer_network_peering) < 63
 else f"{peer_network_peering[:min(58, len(peer_network_peering))]}-{network_peering_suffix}"
)

pulumi.export("localNetworkName", local_network_name)
pulumi.export("peerNetworkName", peer_network_name)
pulumi.export("localNetworkPeeringName", local_network_peering_name)
pulumi.export("peerNetworkPeeringName", peer_network_peering_name)

package main

import (
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 localNetwork := "your-local-network"
 peerNetwork := "your-peer-network"
 prefix := "your-prefix"

 localNetworkName := reverseString(getNetworkName(localNetwork))
 peerNetworkName := reverseString(getNetworkName(peerNetwork))

 localNetworkPeering := prefix + "-" + localNetworkName + "-" + peerNetworkName
 peerNetworkPeering := prefix + "-" + peerNetworkName + "-" + localNetworkName
 networkPeeringSuffix := "peering"

 localNetworkPeeringName := condenseName(localNetworkPeering, networkPeeringSuffix)
 peerNetworkPeeringName := condenseName(peerNetworkPeering, networkPeeringSuffix)

 ctx.Export("localNetworkName", pulumi.String(localNetworkName))
 ctx.Export("peerNetworkName", pulumi.String(peerNetworkName))
 ctx.Export("localNetworkPeeringName", pulumi.String(localNetworkPeeringName))
 ctx.Export("peerNetworkPeeringName", pulumi.String(peerNetworkPeeringName))

 return nil
 })
}

func getNetworkName(network string) string {
 parts := strings.Split(network, "/")
 return parts[len(parts)-1]
}

func reverseString(s string) string {
 runes := []rune(s)
 for i, j := 0, len(runes)-1; i < j; i, j = i+1, j-1 {
 runes[i], runes[j] = runes[j], runes[i]
 }
 return string(runes)
}

func condenseName(name string, suffix string) string {
 if len(name) < 63 {
 return name
 }
 return name[:min(58, len(name))] + "-" + suffix
}

func min(a, b int) int {
 if a < b {
 return a
 }
 return b
}

using System;
using System.Linq;
using Pulumi;

class Program
{
 static Task<int> Main() => Deployment.RunAsync<MyStack>();

 class MyStack : Stack
 {
 public MyStack()
 {
 var localNetwork = "your-local-network";
 var peerNetwork = "your-peer-network";
 var prefix = "your-prefix";

 var localNetworkName = ReverseString(GetNetworkName(localNetwork));
 var peerNetworkName = ReverseString(GetNetworkName(peerNetwork));

 var localNetworkPeering = $"{prefix}-{localNetworkName}-{peerNetworkName}";
 var peerNetworkPeering = $"{prefix}-{peerNetworkName}-{localNetworkName}";
 var networkPeeringSuffix = "peering";

 var localNetworkPeeringName = CondenseName(localNetworkPeering, networkPeeringSuffix);
 var peerNetworkPeeringName = CondenseName(peerNetworkPeering, networkPeeringSuffix);

 this.Export("localNetworkName", localNetworkName);
 this.Export("peerNetworkName", peerNetworkName);
 this.Export("localNetworkPeeringName", localNetworkPeeringName);
 this.Export("peerNetworkPeeringName", peerNetworkPeeringName);
 }

 private string GetNetworkName(string network)
 {
 var parts = network.Split('/');
 return parts[parts.Length - 1];
 }

 private string ReverseString(string s)
 {
 var chars = s.ToCharArray();
 Array.Reverse(chars);
 return new string(chars);
 }

 private string CondenseName(string name, string suffix)
 {
 if (name.Length < 63) return name;
 return name.Substring(0, Math.Min(58, name.Length)) + "-" + suffix;
 }
 }
}

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.resources.Stack;

import java.util.Arrays;

public class Program {
 public static void main(String[] args) {
 Pulumi.run(Program::stack);
 }

 public static void stack(Context ctx) {
 var localNetwork = "your-local-network";
 var peerNetwork = "your-peer-network";
 var prefix = "your-prefix";

 var localNetworkName = reverseString(getNetworkName(localNetwork));
 var peerNetworkName = reverseString(getNetworkName(peerNetwork));

 var localNetworkPeering = String.format("%s-%s-%s", prefix, localNetworkName, peerNetworkName);
 var peerNetworkPeering = String.format("%s-%s-%s", prefix, peerNetworkName, localNetworkName);
 var networkPeeringSuffix = "peering";

 var localNetworkPeeringName = condenseName(localNetworkPeering, networkPeeringSuffix);
 var peerNetworkPeeringName = condenseName(peerNetworkPeering, networkPeeringSuffix);

 ctx.export("localNetworkName", Output.of(localNetworkName));
 ctx.export("peerNetworkName", Output.of(peerNetworkName));
 ctx.export("localNetworkPeeringName", Output.of(localNetworkPeeringName));
 ctx.export("peerNetworkPeeringName", Output.of(peerNetworkPeeringName));
 }

 private static String getNetworkName(String network) {
 var parts = network.split("/");
 return parts[parts.length - 1];
 }

 private static String reverseString(String s) {
 var chars = s.toCharArray();
 for (int i = 0, j = chars.length - 1; i < j; i++, j--) {
 char temp = chars[i];
 chars[i] = chars[j];
 chars[j] = temp;
 }
 return new String(chars);
 }

 private static String condenseName(String name, String suffix) {
 if (name.length() < 63) return name;
 return name.substring(0, Math.min(58, name.length())) + "-" + suffix;
 }
}

Pitfalls of HCL’s `try` Function

HCL’s try function can be problematic because it evaluates each argument until one does not produce an error. While it seems useful for navigating complex data structures, it often masks underlying issues by ignoring errors rather than addressing them.

This behavior can make debugging difficult because it can hide the real reasons behind failures, leaving configurations brittle and error-prone. Here is an example of the overuse of the try function

platform = try(each.value.platform, var.eks_managed_node_group_defaults.platform, "linux")
cluster_endpoint = try(time_sleep.this[0].triggers["cluster_endpoint"], "")
cluster_auth_base64 = try(time_sleep.this[0].triggers["cluster_certificate_authority_data"], "")
cluster_service_ipv4_cidr = var.cluster_service_ipv4_cidr
cluster_ip_family = var.cluster_ip_family
cluster_service_cidr = try(time_sleep.this[0].triggers["cluster_service_cidr"], "")
enable_bootstrap_user_data = try(each.value.enable_bootstrap_user_data, var.eks_managed_node_group_defaults.enable_bootstrap_user_data, false)
pre_bootstrap_user_data = try(each.value.pre_bootstrap_user_data, var.eks_managed_node_group_defaults.pre_bootstrap_user_data, "")
post_bootstrap_user_data = try(each.value.post_bootstrap_user_data, var.eks_managed_node_group_defaults.post_bootstrap_user_data, "")
bootstrap_extra_args = try(each.value.bootstrap_extra_args, var.eks_managed_node_group_defaults.bootstrap_extra_args, "")
user_data_template_path = try(each.value.user_data_template_path, var.eks_managed_node_group_defaults.user_data_template_path, "")
cloudinit_pre_nodeadm = try(each.value.cloudinit_pre_nodeadm, var.eks_managed_node_group_defaults.cloudinit_pre_nodeadm, [])
cloudinit_post_nodeadm = try(each.value.cloudinit_post_nodeadm, var.eks_managed_node_group_defaults.cloudinit_post_nodeadm, [])

Switching to Pulumi provides a clearer, more maintainable approach:

import * as pulumi from "@pulumi/pulumi";

// Example configuration using TypeScript for better error handling
const config = new pulumi.Config();
const platform = config.get("platform") || "linux"; // Default to "linux" if undefined

// Output using straightforward TypeScript logic
pulumi.output({ platform });

import pulumi

# Example configuration using Python
config = pulumi.Config()
platform = config.get("platform") or "linux" # Default to "linux" if undefined

# Output using straightforward Python logic
pulumi.export("platform", platform)

package main

import (
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 platform, ok := ctx.GetConfig("platform")
 if !ok {
 platform = "linux" // Default to "linux" if undefined
 }

 ctx.Export("platform", pulumi.String(platform))
 return nil
 })
}

using Pulumi;

class Program
{
 static Task<int> Main() => Pulumi.Deployment.RunAsync(() =>
 {
 var config = new Config();
 var platform = config.Get("platform") ?? "linux"; // Default to "linux" if undefined

 return new Dictionary<string, object?>
 {
 { "platform", platform }
 };
 });
}

import com.pulumi.Config;
import com.pulumi.Pulumi;

import java.util.HashMap;
import java.util.Map;

public class Main {
 public static void main(String[] args) {
 Pulumi.run(ctx -> {
 Config config = Config.auto();
 String platform = config.get("platform").orElse("linux"); // Default to "linux" if undefined

 Map<String, Object> result = new HashMap<>();
 result.put("platform", platform);

 ctx.export("result", result);
 });
 }
}

This Pulumi example shows how to handle configuration values in a more straightforward way, avoiding the need for complex try functions which potentially sweep errors under the rug. By using standard programming constructs, you can improve the readability and maintainability of your infrastructure code.

In Conclusion

In conclusion, Pulumi’s approach to infrastructure as code offers significant advantages over HCL, positioning it as a more developer-friendly and future-proof solution. By leveraging general-purpose programming languages, Pulumi addresses many of the limitations and complexities inherent in HCL.

Pulumi’s use of languages like TypeScript, Python, and Go brings several key benefits:

Larger ecosystem and community support, providing better tooling, debugging capabilities, and resources.
Familiar syntax and constructs for developers, reducing the learning curve and improving code readability.
Greater flexibility and expressiveness, allowing for more elegant solutions to complex infrastructure challenges.
Better integration with existing development workflows and practices.
Improved collaboration between development and operations teams, helping to break down silos.

The examples provided throughout this comparison demonstrate how Pulumi’s approach leads to cleaner, more intuitive code. Complex operations that require convoluted HCL expressions or multiple custom resources can often be accomplished with straightforward programming constructs in Pulumi.

Moreover, Pulumi’s model is inherently more adaptable to future changes in cloud technologies and programming paradigms. As new languages or cloud services emerge, Pulumi can easily incorporate them, ensuring that your infrastructure-as-code practices remain current and effective.

For organizations looking to modernize their infrastructure management, streamline their DevOps processes, and empower their development teams, Pulumi presents the only alternative to HCL. Its ability to unify application and infrastructure code under a single programming model offers a path to more efficient, maintainable, and scalable cloud engineering practices.

By choosing Pulumi, teams can enjoy the benefits of true programming languages for infrastructure, leading to more robust, flexible, and developer-friendly cloud deployments.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Interested in migrating? Our cloud experts can help. Learn more about Pulumi’s Migration Services.

Next-level IaC: How Pulumi Supports Your API Economy Strategy

Engin Diri — Wed, 15 May 2024 00:00:00 +0000

When I am talking with community members, who are not using Pulumi yet, I often get asked what would be a good way to include their Infrastructure as Code (IaC) into existing software like a REST API. And my answer is always the same: Use our Pulumi Automation API.

With the Pulumi Automation API you can include Pulumi IaC into your existing software, and this for any of the Pulumi supported programming languages like TypeScript, Python, Go, or C#. This gives you a greater flexibility and control, which you will not have with other IaC tools like CloudFormation or Terraform.

This opens up many doors of opportunity to expose your services as an API, whether internally to your organization or externally to your potential customers. This opportunity for innovation fuels what is known as the API Economy.

What is API Economy?

As a matter of fact: — APIs enable innovation.

Over 40% of large organizations have 250+ APIs. And 71% plan to use even more APIs this year.

API Economy is a term that is used to describe the way that organizations are using APIs to create new business models and revenue streams. APIs are the building blocks of modern software development, and they allow developers to build on top of existing services and create new services that are greater than the sum of their parts.

Major international corporations leveraging the API economy through robust API strategies include Google, Amazon, Stripe, Twilio, Slack, and many other renowned brands. Google’s initial Maps service, for instance, has evolved into a comprehensive platform offering Maps, Routes, and Places APIs and SDKs. These tools enable companies to incorporate both dynamic and static maps into their applications and websites, offer navigational guidance, access 360° Street View images, utilize geolocation services, and more. This transformation exemplifies the concept of value multiplication in the API economy, as described by Gartner Vice President Kristin Moyer:

“The API economy is an enabler for turning a business or organisation into a platform. Platforms multiply value creation because they enable business ecosystems inside and outside of the enterprise to consummate matches among users and facilitate the creation and exchange of goods, services and social currency so that all participants are able to capture value.”

What is Pulumi Automation API in a nutshell?

Pulumi Automation API is a set of SDKs that allow you to interact with your Pulumi stacks programmatically. This means that you can use the Automation API to create, update, and delete stacks, as well as to query their current state. The Automation API is built on top of the Pulumi engine, which means that it is fully compatible with all of the existing Pulumi providers and resources.

And it is this capability that allows you to build your own Service API on top of Pulumi. We can now build a REST or gRPC API endpoint that, depending on the request parameters, will create, update, or delete a Pulumi stack. This API can then be exposed to the rest of the organization, allowing developers to interact with their infrastructure in a programmatic way.

How to build your own Service API with Pulumi Automation API?

Let’s take a look at how you can build your own Service API with Pulumi Automation API.

For this purpose, we will build two basic REST services written in TypeScript and in Python.

import express from "express"
import * as dotevnv from "dotenv"
import cors from "cors"
import helmet from "helmet"
import {fullyQualifiedStackName, RemoteWorkspace} from "@pulumi/pulumi/automation";

dotevnv.config()

if (!process.env.PORT) {
 console.log(`No port value specified...`)
}

const PORT = parseInt(process.env.PORT as string, 10)

const app = express()

app.use(express.json())
app.use(express.urlencoded({extended: true}))
app.use(cors())
app.use(helmet())

// Define the payload interface
interface ServicePayload {
 name: string;
 org: string;
 project: string;
 stack: string;
 region: string;
}

// Define a helper function to set up the Pulumi program
async function setUpPulumiProgram(payload: ServicePayload) {
 const org = payload.org;
 const project = payload.project;
 const stackName = fullyQualifiedStackName(org, project, payload.stack);
 const awsRegion = payload.region;

 // Using the Pulumi Automation API to create or select a stack in the remote workspace
 // and set the stack's configuration and secrets. This will use Pulumi Deployments to
 // deploy the stack to the specified AWS region.
 return await RemoteWorkspace.createOrSelectStack({
 stackName,
 url: "https://github.com/pulumi/examples.git",
 branch: "refs/heads/master",
 projectPath: project,
 }, {
 envVars: {
 AWS_REGION: awsRegion,
 AWS_ACCESS_KEY_ID: process.env.AWS_ACCESS_KEY_ID ?? "",
 AWS_SECRET_ACCESS_KEY: {secret: process.env.AWS_SECRET_ACCESS_KEY ?? ""},
 AWS_SESSION_TOKEN: {secret: process.env.AWS_SESSION_TOKEN ?? ""},
 },
 });
}

// Define the API endpoint for creating a new stack in the remote workspace.
app.post('/', async (req, res) => {
 const payload: ServicePayload = req.body;

 if ((payload.name) || !payload.org || !payload.project || !payload.stack || !payload.region) {
 res.status(400).send('Name, org, project, stack and region are required');
 return;
 }
 // Create a new stack in the remote workspace and set the stack's configuration and secrets.
 const stack = await setUpPulumiProgram(payload);

 // Deploy the stack to the specified AWS region. We'll print the stack's outputs to the console.
 const upRes = await stack.up({onOutput: console.log});
 res.status(200).send(`url: ${upRes.outputs.websiteUrl.value}`);
});

// Define the API endpoint for destroying a stack in the remote workspace.
app.delete('/', async (req, res) => {
 const {name, org, project, stack, region} = req.query;

 if (!name || !org || !project || !stack || !region) {
 res.status(400).send('Name, org, project, stack, and region are required');
 return;
 }

 const payload: ServicePayload = {
 name: name as string,
 org: org as string,
 project: project as string,
 stack: stack as string,
 region: region as string
 }

 // Define the stack and set the stack's configuration and secrets.
 const prog = await setUpPulumiProgram(payload);
 // Destroy the stack in the remote workspace. We'll print the stack's outputs to the console.
 await prog.destroy({onOutput: console.log});
 res.status(200).send('Stack destroyed');
});


app.listen(PORT, () => {
 console.log(`Server is listening on port ${PORT}`)
})

import os
import pulumi.automation as auto
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()


# Define the payload that the API will receive. This payload will contain the name of the stack
# to create, the organization and project to use, the stack name, and the AWS region to deploy the stack to.
class ServicePayload(BaseModel):
 name: str
 org: str
 project: str
 stack: str
 region: str


# Define a helper function that will create a new stack in the remote workspace and set the stack's configuration
def setup_pulumi_program(payload: ServicePayload):
 org = payload.org
 project = payload.project
 stack_name = auto.fully_qualified_stack_name(org, project, payload.stack)
 aws_region = payload.region

 # Using the Pulumi Automation API to create or select a stack in the remote workspace
 # and set the stack's configuration and secrets. This will use Pulumi Deployments to
 # deploy the stack to the specified AWS region.
 return auto.create_or_select_remote_stack_git_source(
 stack_name=stack_name,
 url="https://github.com/pulumi/examples.git",
 branch="refs/heads/master",
 project_path=project,
 opts=auto.RemoteWorkspaceOptions(
 env_vars={
 "AWS_REGION": aws_region,
 "AWS_ACCESS_KEY_ID": os.environ["AWS_ACCESS_KEY_ID"],
 "AWS_SECRET_ACCESS_KEY": auto.Secret(os.environ["AWS_SECRET_ACCESS_KEY"]),
 "AWS_SESSION_TOKEN": auto.Secret(os.environ["AWS_SESSION_TOKEN"]),
 },
 ),
 )


# Define the API endpoints for creating and deleting a stack in the remote workspace.
@app.post("/")
async def create(payload: ServicePayload):
 # Create a new stack in the remote workspace and set the stack's configuration and secrets.
 stack = setup_pulumi_program(payload)
 # Deploy the stack to the specified AWS region. We'll print the stack's outputs to the console.
 up_res = stack.up(on_output=print)
 return {"url": up_res.outputs['websiteUrl'].value}


# Define the API endpoint for deleting a stack in the remote workspace.
@app.delete("/")
async def delete(name: str, org: str, project: str, stack: str, region: str):
 payload = ServicePayload(name=name, org=org, project=project, stack=stack, region=region)
 # Define the stack and set the stack's configuration and secrets.
 stack = setup_pulumi_program(payload)
 # Destroy the stack in the remote workspace. We'll print the stack's outputs to the console.
 stack.destroy(on_output=print)
 return {"message": "Stack destroyed"}

Let us run the above Python example in a FastAPI server to see how the Automation API works in practice.

Let’s start by creating a new Python file called main.py and pasting the above code into it. Next, we need to install the required dependencies:

python3 -m venv ven
source venv/bin/activate

Create a new file called requirements.txt and paste the following dependencies into it:

pulumi
fastapi
pydantic

We need to install the dependencies using the following command:

pip install -r requirements.txt --force-reinstall

Finally, we can run the FastAPI server using the following command:

fastapi dev main.py

This will start the FastAPI server on port 8000.

INFO Using path main.py
INFO Resolved absolute path /private/tmp/automation-api-example/python/main.py
INFO Searching for package file structure from directories with __init__.py files
INFO Importing from /private/tmp/automation-api-example/python

 ╭─ Python module file ─╮
 │ │
 │ 🐍 main.py │
 │ │
 ╰──────────────────────╯

INFO Importing module main
INFO Found importable FastAPI app

 ╭─ Importable FastAPI app ─╮
 │ │
 │ from main import app │
 │ │
 ╰──────────────────────────╯

INFO Using import string main:app

 ╭────────── FastAPI CLI - Development mode ───────────╮
 │ │
 │ Serving at: http://127.0.0.1:8000 │
 │ │
 │ API docs: http://127.0.0.1:8000/docs │
 │ │
 │ Running in development mode, for production use: │
 │ │
 │ fastapi run │
 │ │
 ╰─────────────────────────────────────────────────────╯

INFO: Will watch for changes in these directories: ['/private/tmp/automation-api-example/python']
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: Started reloader process [55849] using WatchFiles
INFO: Started server process [55851]
INFO: Waiting for application startup.
INFO: Application startup complete.

To test the REST services, you can use a tool like curl or Postman. Here is an example of how you can create a new stack using curl:

curl -X POST -H "Content-Type: application/json" -d '{"name": "myexample", "org": "myorg", "project": "myproject", "stack": "mystack", "region": "us-west-2"}' http://localhost:3000

In the logs of the FastAPI server, you should see following output from the Automation API:

INFO: Create stack
Preparing deployment...

View Live: https://app.pulumi.com/dirien/aws-ts-s3-folder/dev/deployments/3

Setup:
Preparing environment
Pulling container image "pulumi/pulumi:3.115.2"
3.115.2: Pulling from pulumi/pulumi
Digest: sha256:a7b9022d7c1a40d81adeca30a21eb1cd49add474e3de923ade663db046e7ee1a
Status: Downloaded newer image for pulumi/pulumi:3.115.2
Starting container
Container created (6b92928bdcdfe6e54286dcdcb622480004f18c98b23b5b60775f71959a423708)
Container started (6b92928bdcdfe6e54286dcdcb622480004f18c98b23b5b60775f71959a423708)

Download deployment executor:
$ curl --retry 5 -H "Authorization: token $PULUMI_ACCESS_TOKEN" "$PULUMI_SERVICE_URL/deployments/executor" -o /pulumi-deploy-executor.gz && gzip -d /pulumi-deploy-executor.gz && chmod +x /pulumi-deploy-executor
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
100 15.9M 0 15.9M 0 0 42.2M 0 --:--:-- --:--:-- --:--:-- 42.4M

Get source:
$ /pulumi-deploy-executor getsource git --repoURL="https://github.com/pulumi/examples.git" --workDir="/deployment" --branch="refs/heads/master" --repoDir="aws-ts-s3-folder"
Successfully cloned: https://github.com/pulumi/examples.git

Download dependencies:
$ /pulumi-deploy-executor dependencies download --workDir="/deployment/aws-ts-s3-folder"
npm WARN deprecated @opentelemetry/api-metrics@0.32.0: Please use @opentelemetry/api >= 1.3.0

added 294 packages, and audited 295 packages in 14s

43 packages are looking for funding
 run `npm fund` for details

found 0 vulnerabilities
npm notice
npm notice New major version of npm available! 9.6.7 -> 10.7.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.7.0>
npm notice Run `npm install -g npm@10.7.0` to update!
npm notice

pulumi update:
$ /pulumi-deploy-executor pulumi update --stackIdentity="dirien/aws-ts-s3-folder/dev" --workDir="/deployment/aws-ts-s3-folder"
Updating (dev)

View Live: https://app.pulumi.com/dirien/aws-ts-s3-folder/dev/updates/3


 + pulumi:pulumi:Stack aws-ts-s3-folder-dev creating (0s)
 + aws:s3:Bucket s3-website-bucket creating (0s)
 + aws:s3:Bucket s3-website-bucket created (4s)
 + aws:s3:BucketObject favicon.png creating (0s)
 + aws:s3:BucketObject index.html creating (0s)
 + aws:s3:BucketPublicAccessBlock public-access-block creating (0s)
 + aws:s3:BucketObject index.html created (0.98s)
 + aws:s3:BucketObject favicon.png created (1s)
 + aws:s3:BucketPublicAccessBlock public-access-block created (1s)
 + aws:s3:BucketPolicy bucketPolicy creating (0s)
 + aws:s3:BucketPolicy bucketPolicy created (0.83s)
 + pulumi:pulumi:Stack aws-ts-s3-folder-dev created (18s)
Outputs:
 bucketName: "s3-website-bucket-d55e9bd"
 websiteUrl: "s3-website-bucket-d55e9bd.s3-website.eu-central-1.amazonaws.com"

Resources:
 + 6 created

Duration: 20s

Result: succeeded
INFO: 127.0.0.1:49220 - "POST / HTTP/1.1" 200 OK

You should see the URL of the created infrastructure in the response from the FastAPI server. If you head over to the Pulumi console, you should see the created stack and the resources that were created and also that Pulumi Deployments were used to deploy the stack to the specified AWS region.

And here is an example of how you can delete a stack using curl:

curl -X DELETE "http://localhost:8000/?name=myexample&org=myorg&project=myproject&stack=mystack&region=us-west-2"

In the logs of the FastAPI server, you should see the following output from the Automation API:

INFO: Deleting stack
Preparing deployment...

View Live: https://app.pulumi.com/dirien/aws-ts-s3-folder/dev/deployments/4

Setup:
Preparing environment
Pulling container image "pulumi/pulumi:3.115.2"
3.115.2: Pulling from pulumi/pulumi
Digest: sha256:a7b9022d7c1a40d81adeca30a21eb1cd49add474e3de923ade663db046e7ee1a
Status: Downloaded newer image for pulumi/pulumi:3.115.2
Starting container
Container created (26af871d4e26a6dab5656a1c8dc8a7e87bf247f42bde77e2269c269f15054e73)
Container started (26af871d4e26a6dab5656a1c8dc8a7e87bf247f42bde77e2269c269f15054e73)

Download deployment executor:
$ curl --retry 5 -H "Authorization: token $PULUMI_ACCESS_TOKEN" "$PULUMI_SERVICE_URL/deployments/executor" -o /pulumi-deploy-executor.gz && gzip -d /pulumi-deploy-executor.gz && chmod +x /pulumi-deploy-executor
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
100 15.9M 0 15.9M 0 0 43.5M 0 --:--:-- --:--:-- --:--:-- 43.5M

Get source:
$ /pulumi-deploy-executor getsource git --repoURL="https://github.com/pulumi/examples.git" --workDir="/deployment" --branch="refs/heads/master" --repoDir="aws-ts-s3-folder"
Successfully cloned: https://github.com/pulumi/examples.git

Download dependencies:
$ /pulumi-deploy-executor dependencies download --workDir="/deployment/aws-ts-s3-folder"
npm WARN deprecated @opentelemetry/api-metrics@0.32.0: Please use @opentelemetry/api >= 1.3.0

added 294 packages, and audited 295 packages in 14s

43 packages are looking for funding
 run `npm fund` for details

found 0 vulnerabilities
npm notice
npm notice New major version of npm available! 9.6.7 -> 10.7.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.7.0>
npm notice Run `npm install -g npm@10.7.0` to update!
npm notice

pulumi destroy:
$ /pulumi-deploy-executor pulumi destroy --stackIdentity="dirien/aws-ts-s3-folder/dev" --workDir="/deployment/aws-ts-s3-folder"
Destroying (dev)

View Live: https://app.pulumi.com/dirien/aws-ts-s3-folder/dev/updates/4


 - aws:s3:BucketPolicy bucketPolicy deleting (0s)
 - aws:s3:BucketPolicy bucketPolicy deleted (1s)
 - aws:s3:BucketPublicAccessBlock public-access-block deleting (0s)
 - aws:s3:BucketObject favicon.png deleting (0s)
 - aws:s3:BucketObject index.html deleting (0s)
 - aws:s3:BucketObject index.html deleted (0.61s)
 - aws:s3:BucketObject favicon.png deleted (0.73s)
 - aws:s3:BucketPublicAccessBlock public-access-block deleted (0.77s)
 - aws:s3:Bucket s3-website-bucket deleting (0s)
 - aws:s3:Bucket s3-website-bucket deleted (0.79s)
 - pulumi:pulumi:Stack aws-ts-s3-folder-dev deleting (0s)
 - pulumi:pulumi:Stack aws-ts-s3-folder-dev deleted (0.10s)
Outputs:
 - bucketName: "s3-website-bucket-d55e9bd"
 - websiteUrl: "s3-website-bucket-d55e9bd.s3-website.eu-central-1.amazonaws.com"

Resources:
 - 6 deleted

Duration: 7s

The resources in the stack have been deleted, but the history and configuration associated with the stack are still maintained.
If you want to remove the stack completely, run `pulumi stack rm dev`.
Result: succeeded
INFO: 127.0.0.1:51509 - "DELETE /?name=my-test&org=dirien&project=aws-ts-s3-folder&stack=dev&region=eu-central-1 HTTP/1.1" 200 OK

Next Steps

Now that we have our basic REST services, we can deploy them on a server (using a service like AWS Lambda, Google Cloud Functions, or Azure Functions) and expose them to the rest of the organization via an API Gateway. In parallel you can publish release notes and documentation to your organization, so that they can start using your new Service API.

One way could be to create an OpenAPI specification for your Service API and use a tool like Swagger to generate a client library for your organization. This way, developers can interact with your Service API in a programmatic way, without having to write any code.

Here is an example of an OpenAPI specification for our Service API:

openapi: 3.0.0
info:
 title: My Organization Service API
 description: API for creating and destroying a specific piece of infrastructure
 version: 1.0.0
servers:
- url: 'http://localhost:8000/'
 description: Development server
paths:
 /:
 post:
 summary: Create Infrastructure
 operationId: createInfrastructure
 requestBody:
 required: true
 content:
 application/json:
 schema:
 $ref: '#/components/schemas/ServicePayload'
 responses:
 '200':
 description: Infrastructure successfully created
 content:
 application/json:
 schema:
 type: object
 properties:
 url:
 type: string
 description: URL of the created infrastructure
 '400':
 description: Bad request if the input payload is incorrect
 '500':
 description: Internal server error
 delete:
 summary: Destroy Infrastructure
 operationId: destroyInfrastructure
 parameters:
 - in: query
 name: name
 schema:
 type: string
 required: true
 description: Name of the service
 - in: query
 name: org
 schema:
 type: string
 required: true
 description: Organization in Pulumi
 - in: query
 name: project
 schema:
 type: string
 required: true
 description: Project name in Pulumi
 - in: query
 name: stack
 schema:
 type: string
 required: true
 description: Stack name in Pulumi
 - in: query
 name: region
 schema:
 type: string
 required: true
 description: AWS region where the service will be deployed
 responses:
 '200':
 description: Infrastructure successfully destroyed
 content:
 application/json:
 schema:
 type: object
 properties:
 message:
 type: string
 description: Confirmation message of the destruction
 '400':
 description: Bad request if the required parameters are missing
 '500':
 description: Internal server error
components:
 schemas:
 ServicePayload:
 type: object
 required:
 - name
 - org
 - project
 - stack
 - region
 properties:
 name:
 type: string
 description: Name of the service
 org:
 type: string
 description: Organization in Pulumi
 project:
 type: string
 description: Project name in Pulumi
 stack:
 type: string
 description: Stack name in Pulumi
 region:
 type: string
 description: AWS region where the service will be deployed

Navigate to the Swagger Editor and paste the above OpenAPI specification to see how it looks.

Conclusion

In this blog post, we’ve explored how Pulumi’s Automation API can be used to build and expose a Service API within your organization. By embedding Pulumi as a library in any application, you can drive your Infrastructure as Code (IaC) deployments directly through code. This sets Pulumi apart from other tools like Terraform, which relies on static configuration files and a command-line interface.

Additionally, we touched on the concept of the API economy, where APIs are leveraged to connect services, accelerate product delivery, and drive business value through the possibilities of creating new touchpoints. The Pulumi Automation API is the perfect piece of technology, which makes it now possible to define, deploy, and manage infrastructure programmatically as part of your API strategy.

I hope this blog post has inspired you with ideas on how to take your IaC to the next level, making it more accessible and practical for your organization.

Next Steps

If you haven’t already, install Pulumi today, and follow our self-directed Getting Started guides to learn more about making the most of Pulumi’s next-level infrastructure management features at your organization.

To learn more, you can watch the following video which provides a high level overview of how Pulumi works:

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}">

Pulumi Cloud

The Pulumi Cloud is a fully managed service that helps you adopt Pulumi’s open source SDK with ease. It provides built-in state and secrets management, integrates with source control and CI/CD, and offers a web console and API that make it easier to visualize and manage infrastructure. It is free for individual use, with features available for teams.

Create an Account

Next-level IaC: Powering Up Pulumi with Package Ecosystems

Troy Howard — Mon, 13 May 2024 00:00:00 +0000

Every experienced tech professional I know has a programming language they love. But is it the syntax and symbols that make it so loveable? Not really. It’s the community and package ecosystem surrounding the language that makes a real impact on your heart… and on your productivity!

If we look at some of the biggest success stories in tech — Python, Node.js, Ruby, Perl, and Go — the common thread between all of them is an extensive ecosystem of packages, libraries, modules (or whatever you decide to call them… ahem, Gems?!). A great language will allow you to build anything you can imagine, but a great ecosystem will have already written it for you, and made it available in a convenient to install-and-use package.

One of the amazing things about Pulumi is that it is built around general-purpose programming languages, and that means your Pulumi programs have access to the entire ecosystem of packages that come with each language. This is a stark difference between Pulumi and other infrastructure automation tools that use proprietary domain-specific languages with not much in the way of community around them. Some tools might allow you to write custom code, but they certainly don’t make it convenient, and still… you have to write it yourself, which just adds so much overhead to a project where your core concern isn’t writing that custom function, but rather shipping your own product on the infrastructure you are trying to automate.

The flexibility that Pulumi gives you, and the access to these powerful libraries of code, means your productivity can skyrocket when you get to write in your favorite language, using your preferred tools and the package ecosystem you already know, love, and rely on every day.

Let’s have a look at a concrete example where this package ecosystem can get you out of a tough spot.

Use Case: Determining S3 MIME types on-the-fly

A popular example of cloud infrastructure automation is creating an S3 bucket and pushing some files to it. This is a common task that many of us have had to do, and most infrastructure automation tools have a way to accomplish this.

Here’s one way to do that in Pulumi:

import * as pulumi from '@pulumi/pulumi';
import * as aws from '@pulumi/aws';

// Create an S3 bucket where we will upload the file.
const bucket = new aws.s3.Bucket("my-bucket", {
 // You can apply additional bucket settings here.
});

const filePath = "path/to/your/local/file.txt"; // Replace with the path to the file you want to upload

// Upload the file to the S3 bucket.
const bucketObject = new aws.s3.BucketObject("my-bucket-object", {
 bucket: bucket, // Reference to the bucket created above.
 key: "file.txt", // This will be the file's name in the bucket.
 source: pulumi.FileAsset(filePath), // Contents of the file.
 contentType: "text/plain", // MIME-type of the file. Adjust if you upload a different type of file.
});

// Export the name of the bucket
export const bucketName = bucket.id;

// Export the URL of the file in the S3 bucket
export const objectUrl = pulumi.interpolate`s3://${bucket.id}/${bucketObject.key}`;

import pulumi
import pulumi_aws as aws

# Create an S3 bucket
bucket = aws.s3.Bucket('my-bucket')

# The path to the file to upload to S3
file_path = "path/to/your/local/file.txt" # Replace with the path to the file you want to upload

# Upload the file to the S3 bucket
bucket_object = aws.s3.BucketObject('my-bucket-object',
 bucket=bucket.id, # Reference to the bucket created above.
 key='file.txt', # This will be the file's name in the bucket.
 source=pulumi.FileAsset(file_path),
 content_type='text/plain') # MIME-type of the file. Adjust if you upload a different type of file.

# Export the name of the bucket
pulumi.export('bucket_name', bucket.id)

# Export the URL of the file in the S3 bucket
pulumi.export('object_url', pulumi.Output.concat('s3://', bucket.id, '/', bucket_object.key))

However, something that stands out as a weak link here is the line:

contentType: "text/plain", // MIME-type of the file. Adjust if you upload a different type of file.

content_type='text/plain') # MIME-type of the file. Adjust if you upload a different type of file.

Yeah, hard-coding the MIME type for every file I need to upload is a not going to scale. In fact, I’m probably going to get a random directory full of files I need to iterate over, each with different file types, and it will change every time it runs, and we won’t know what type of file it is ahead of time.

Packages to the rescue!! Turns out that determining the MIME type of a file only involves importing a community-created module like mime-types in TypeScript, or using the mimetypes module from Python’s standard library, and a single function call to guess the MIME type:

import * as mime from 'mime-types'

// Detect the MIME type of the file
const mimeType = mime.lookup('path/to/your/local/file.txt');

import mimetypes

# Detect the MIME type of the file
mime_type, _ = mimetypes.guess_type('path/to/your/local/file.txt')

So, to work that into our Pulumi program from above, we only need to make a few small edits:

import * as pulumi from '@pulumi/pulumi';
import * as aws from '@pulumi/aws';
import * as mime from 'mime-types';

// Create an S3 bucket where we will upload the file.
const bucket = new aws.s3.Bucket("my-bucket", {
 // You can apply additional bucket settings here.
});

const filePath = "path/to/your/local/file.txt"; // Replace with the path to the file you want to upload

// Upload the file to the S3 bucket.
const bucketObject = new aws.s3.BucketObject("my-bucket-object", {
 bucket: bucket, // Reference to the bucket created above.
 key: "file.txt", // This will be the file's name in the bucket.
 source: pulumi.FileAsset(filePath), // Contents of the file.
 contentType: mime.lookup(filePath) || "application/octet-stream", // MIME-type of the file
});

// Export the name of the bucket
export const bucketName = bucket.id;

// Export the URL of the file in the S3 bucket
export const objectUrl = pulumi.interpolate`s3://${bucket.id}/${bucketObject.key}`;

import pulumi
import pulumi_aws as aws
import mimetypes

# Create an S3 bucket
bucket = aws.s3.Bucket('my-bucket')

# The path to the file to upload to S3
file_path = "path/to/your/local/file.txt" # Replace with the path to the file you want to upload

# Detect the MIME type of the file
mime_type, _ = mimetypes.guess_type(file_path)

# Upload the file to the S3 bucket
bucket_object = aws.s3.BucketObject('my-bucket-object',
 bucket=bucket.id, # Reference to the bucket created above.
 key='file.txt', # This will be the file's name in the bucket.
 source=pulumi.FileAsset(file_path),
 content_type=mime_type or 'application/octet-stream') # MIME-type of the file

# Export the name of the bucket
pulumi.export('bucket_name', bucket.id)

# Export the URL of the file in the S3 bucket
pulumi.export('object_url', pulumi.Output.concat('s3://', bucket.id, '/', bucket_object.key))

Well that was pretty straightforward, wasn’t it? Trying to do this in another tool might take all day to figure out the right way to implement it, but in Pulumi it worked exactly like you would expect it to, in no time at all.

It’s a real game-changer when you can just quickly grab something complicated like MIME typing from a package. But when you can’t, you’re going to be stuck figuring out how to write that functionality yourself, from scratch, using things like Bash and primitive data types, and then find yourself copy-pasting that stuff all over the place. All the while fully aware of how much time you’re wasting by having to do so. That’s a real pain! It’s the kind of pain most people managing infrastructure have been through before.

The possibilities are endless

What other packages can you imagine using in your infrastructure solutions? Maybe you could use the text-to-image library to generate an image badge for your website? Maybe use the crypto module to encrypt a file before uploading it to S3? Or possibly using octokit to check on the state of something in git before taking the next step in your automation?

Whatever you can do with code in your favorite language, you can do in Pulumi as part of your infrastructure automation. I’m not saying you should use cowsay to generate custom MOTD banners for your ssh logins, I’m just saying you could.

Next Steps

To learn more, you can watch the following video which provides a high level overview of how Pulumi works:

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}">

Pulumi Cloud

Create an Account

Next-level IaC: Drop those wrapper scripts and let your language do that for you

Christian Nunciato — Tue, 30 Apr 2024 00:00:00 +0000

Our users are always telling us (particularly the ones who come to Pulumi from other IaC tools) that being able to use general-purpose languages to manage their infrastructure was a game changer for them.

I know it was for me. As a JavaScript developer, when I discovered Pulumi and saw that I could do pretty much everything I was doing with Terraform but with TypeScript, I was immediately hooked; that’s all it took. Just being able to write my resource declarations in a language I knew well (and that my IDE understood) was huge.

But it’s easy to forget, even when you’ve been using Pulumi for a while, that you aren’t just getting the language alone — you’re getting the full capability of the language’s runtime environment also. Not just JavaScript, but all of Node.js. Not just C#, but everything the .NET framework, runtime, and ecosystem have to offer.

This is quite a big deal, actually. Whereas with other tools, you’re often bound by the limits of the tool’s DSL, with Pulumi, you have no such constraints; you can do just about anything your chosen language can do. That means much more than just declaring long lists of cloud resources — it means unlocking all kinds of capabilities that can help bring your infrastructure-management practice to a whole new level.

In this post, the first of a new series we’re calling Next-level IaC, we’ll show you a few small but useful ways you can take advantage of this reality — first, by spotting some opportunities to simplify your deployments.

Run pre-deployment tasks with Pulumi

Fundamentally, a Pulumi program is just a regular program. That means you can take actions programmatically at runtime — before the Pulumi engine gets going.

Suppose you were tasked with writing the code to deploy and manage a static website. Conceptually, you might break this process up into a few separate steps:

Build the website
Deploy the supporting infrastructure (e.g, the cloud storage, maybe a content-delivery network)
Copy the content of the website into the infrastructure

This you might lead you to reach for Bash to write up a shell script like the one below, which uses Hugo (a popular static-site generator) to build the website, Pulumi to provision the infrastructure, and the AWS CLI to push the built website up into the AWS cloud:

#!/bin/bash

# Build the website.
hugo --destination ./public

# Provision the infrastructure.
pulumi up --yes

# Copy the website content into the infrastructure.
aws s3 sync ./public s3://$(pulumi stack output bucketName)

And this would definitely work. But it’s also unnecessary. You can avoid both the Bash wrapper and the dependency on the AWS CLI by using your chosen language’s built-in support for spawning shell processes:

import * as aws from "@pulumi/aws";
import * as childProcess from "child_process";

// Build the website.
const result = childProcess.execSync("hugo --destination ./public", { stdio: "pipe", cwd: "./www" });

// Provision a storage bucket for the website.
const bucket = new aws.s3.Bucket("bucket", {
 website: {
 indexDocument: "index.html",
 },
});

// Copy the website home page into the bucket.
const homepage = new aws.s3.BucketObject(
 "index.html",
 {
 bucket: bucket.id,
 content: fs.readFileSync("./www/public/index.html", "utf-8"),
 contentType: "text/html",
 acl: "public-read",
 },
);

import pulumi_aws as aws
import subprocess

# Build the website.
result = subprocess.run(
 ["hugo", "--destination", "./public"],
 stdout=subprocess.PIPE,
 cwd="./www",
 check=True,
 shell=True,
)

# Provision a storage bucket for the website.
bucket = aws.s3.Bucket(
 "bucket", website=aws.s3.BucketWebsiteArgs(index_document="index.html")
)

# Copy the website home page into the bucket
homepage = aws.s3.BucketObject(
 "index.html",
 bucket=bucket.id,
 content=open("./www/public/index.html", "r").read(),
 content_type="text/html",
 acl="public-read",
)

Running processes like these synchronously, as above, means they’ll complete before Pulumi starts registering resources — important here because the BucketObject resource needs that public folder to exist for the deployment to succeed. Fortunately it does, so all is well:

$ pulumi up
Updating (dev)
Type Name Plan Info
+ pulumi:pulumi:Stack example-dev create 15 messages
+ ├─ aws:s3:Bucket bucket create
+ └─ aws:s3:BucketObject index.html create
Diagnostics:
pulumi:pulumi:Stack (example-dev):
Start building sites ...
| EN
-------------------+-----
Pages | 4
Paginator pages | 0
Non-page files | 0
Static files | 0
Processed images | 0
Aliases | 0
Sitemaps | 1
Cleaned | 0
Total in 11 ms
Resources:
+ 3 created

Also notice how the example captures the output of the process in result and writes it to the terminal. Standard logging like this tends not to be possible with static languages (at least not without some hackery) because there’s no real way to do it programmatically. Similarly, those that do support spawning processes tend to require them to return strictly formatted JSON strings — which, as the example shows, not all processes do. (Forcing you to have to wrap them with yet more shell scripts.)

Simple stuff, but flexible and powerful. With one line, you’ve kept your deployment process simple, dropping two tools you don’t need thanks to the built-in capabilities of Pulumi and your language of choice.

Fetch data to generate resources dynamically

To build on the static-website scenario, suppose you wanted to retrieve some data from an external source and use it to provision some cloud resources dynamically. For example, you might want to let your marketing team create short URLs on their own for social media posts. For this, you might once again turn to Bash and the AWS CLI to pull those URLs from your corporate CMS (e.g., with curl), either before or after running pulumi, and create the corresponding redirects imperatively using a follow-up shell command of some sort.

Again, a totally reasonable approach. But again, why bother, when you have the full capability of Node.js Node.js Python Go .NET Java Pulumi YAML .NET .NET at fingertips? Just add a few lines to your program to fetch the data and be on your way:

// ...

// Fetch some redirects from a hypothetical CMS.
const redirects = fetch(`${process.env.CMS_ENDPOINT}/redirects.json`)
 .then(response => response.json())
 .then(items =>
 items.forEach((redirect: any, i: number) => {
 // Create an S3 website redirect for each one.
 new aws.s3.BucketObject(
 `redirect-${i}`,
 {
 bucket: bucket.id,
 key: redirect.from,
 websiteRedirect: redirect.to,
 acl: "public-read",
 },
 { dependsOn: [ownershipControls, publicAccess] },
 );
 }),
 );

#...
import json
import os
import requests

# ...

# Fetch some redirects from a hypothetical CMS.
response = requests.get(f"{os.environ['CMS_ENDPOINT']}/redirects.json")
redirects = json.loads(response.text)

# Create an S3 website redirect for each one.
for i, redirect in enumerate(redirects):
 aws.s3.BucketObject(
 f"redirect-{i}",
 bucket=bucket.id,
 key=redirect["from"],
 website_redirect=redirect["to"],
 acl="public-read",
 )

Notice the example parses the response using the language’s built-in JSON support. If your CMS happened to return data in some other format, or required some additional credentials for authorization, no problem — your language and its package ecosystem are going to be there to help you.

Also notice the example reads the source URL directly from the environment. This is typically more cumbersome (if even possible) with other tools, requiring you to name your environment variables in tool-specific ways and then tack on additional blocks of configuration before you’re able to use them. As you can see, Node.js Node.js Python Go .NET Java Pulumi YAML .NET .NET makes this much more convenient.

Finally, notice the redirects are translated into cloud resources using a familiar JavaScript TypeScript Python Go C# Java Pulumi YAML F# VB looping construct. We’ll cover this topic in much more depth in an upcoming post. (Stay tuned!)

Hook the event loop to run tasks after deployment

Finally, you can do nifty things like run tasks programmatically after a Pulumi deployment completes — once again, without ever having to leave the Pulumi program.

If your hypothetical website were sitting behind a CloudFront CDN, for example, you might want some way to clear the CDN’s cache after a deployment. This generally requires some imperative action, like using the AWS CLI to run aws cloudfront create-invalidation sometime later. How might you do something like this with Pulumi, given Pulumi programs exit automatically when their operations complete?

One way — again, since a Pulumi program is just a regular Node.js Node.js Python Go .NET Java Pulumi YAML .NET .NET program — would be to write a JavaScript TypeScript Python Go C# Java Pulumi YAML F# VB function to create an invalidation request, and then ask the Node.js Node.js Python Go .NET Java Pulumi YAML .NET .NET runtime to run that function before exiting.

The mechanics of this vary slightly by language, but it works, and it’s a very handy way to keep all of your deployment logic in one place — again avoiding the need to relegate anything to a Bash script:

// ...
import * as cloudfront from "@aws-sdk/client-cloudfront";

const config = new pulumi.Config("aws");
const region = config.require("region");

// Create a CloudFront distribution for the website.
const cdn = new aws.cloudfront.Distribution("cdn", {
 enabled: true,
 defaultRootObject: "index.html",
 origins: [{
 originId: bucket.arn,
 domainName: bucket.websiteEndpoint,
 // ...
 }],
 // ...
});

function createInvalidation(id: string) {
 // Only invalidate after a deployment.
 if (pulumi.runtime.isDryRun()) {
 console.log("This is a Pulumi preview, so skipping cache invalidation.");
 return;
 }

 process.on("beforeExit", () => {
 const client = new cloudfront.CloudFrontClient({ region });
 const command = new cloudfront.CreateInvalidationCommand({
 DistributionId: id,
 InvalidationBatch: {
 CallerReference: `invalidation-${Date.now()}`,
 Paths: {
 Quantity: 1,
 Items: ["/*"],
 },
 },
 });

 client
 .send(command)
 .then(result => {
 console.log(`Invalidation status for ${id}: ${result.Invalidation?.Status}.`);
 process.exit(0);
 })
 .catch(error => {
 console.error(error);
 process.exit(1);
 });
 });
}

// Register a function to be invoked before the program exits.
cdn.id.apply(id => createInvalidation(id));

The relevant code is the call to process.on("beforeExit"), which registers a function to be invoked just before the program exits using the resolved ID of the distribution. (For more on how the apply() method works, see Inputs and Outputs.) The function returns early for Pulumi previews — no sense clearing the cache if the site hasn’t changed — and uses the AWS SDK for JavaScript to submit the invalidation request to CloudFront, naming it uniquely with a timestamp and logging the result to the console:

# ...
import boto3
import asyncio

def create_invalidation(id):
 # Don't bother invalidating unless it's an actual deployment.
 if pulumi.runtime.is_dry_run():
 print("This is a Pulumi preview, so skipping cache invalidation.")
 return

 client = boto3.client("cloudfront")
 result = client.create_invalidation(
 DistributionId=id,
 InvalidationBatch={
 "CallerReference": f"invalidation-{time.time()}",
 "Paths": {
 "Quantity": 1,
 "Items": ["/*"],
 },
 },
 )

 print(
 f"Cache invalidation for distribution {id}: {result['Invalidation']['Status']}."
 )


# Register a function to be invoked before the program exits.
cdn.id.apply(lambda id: atexit.register(lambda: create_invalidation(id)))

The relevant code is the call to atexit.register(), which registers a function to be invoked just before the program exits using the resolved ID of the distribution. (For more on how the apply() method works, see Inputs and Outputs.) The function returns early for Pulumi previews — no sense clearing the cache if the site hasn’t changed — and uses Boto3 to submit the invalidation request to CloudFront, naming it uniquely with a timestamp and logging the result to the console:

$ pulumi preview
Previewing update (dev)
Type Name Info
pulumi:pulumi:Stack example-dev 16 messages
Diagnostics:
pulumi:pulumi:Stack (example-dev):
...
This is a Pulumi preview, so skipping cache invalidation.

$ pulumi up
Updating (dev)
Type Name Status Info
pulumi:pulumi:Stack example-dev 16 messages
Diagnostics:
pulumi:pulumi:Stack (example-dev):
...
Cache invalidation of distribution EAJOVC1QNGT13: InProgress.

Techniques like these are a great way to eliminate unnecessary scripts, glue code, and other dependencies while keeping your deployment logic clear, well contained, and amenable to change.

Wrapping up

Hopefully this first post has given you a few new ways to think about how to do more with Pulumi and your language of choice. Over the next few weeks, as this series unfolds, we’ll share a lot more, so keep an eye on this space!

You’ll find the full source for the examples above on GitHub:

Happy coding!

Next Steps

To learn more, you can watch the following video which provides a high level overview of how Pulumi works:

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}">

Pulumi Cloud

Create an Account

Pulumi Blog: Hcl

Pulumi for All Your IaC — Including Terraform and HCL

The TL;DR

Terraform/OpenTofu in Pulumi Cloud

HCL Language Support in Pulumi IaC

Builds on Existing Coexist/Convert Capabilities

Pulumi Covered Until Your HashiCorp, an IBM Company, Renewal

Why Snowflake, BMW, and Wiz Switched to Pulumi

Get Started Today

Pulumi vs HCL: Understanding the Language Differences in Infrastructure as Code

Real-World Consequences

Tapping Into Millions of Developers

Pulumi Leverages Rich Ecosystems of General-Purpose Languages

Pulumi Embraces Open-Source Languages and Principles

Developers Onboard Quicker with Pulumi

Enhancing Team Collaboration With Pulumi

Reducing Redundancy and Copy-Paste Challenges

Reduce Need for Custom Tooling

Future-Proofing Your Infrastructure

Making Infrastructure Code Simpler

HCL in the Wild: Practical Comparisons With Pulumi

for_each Loop With Conditional Expression

for_each Loop With Multi-Level Nested Dynamic Blocks

Execute Programming Code With null_resource, local_file and provisioners

If Statements Constructed With count

HCL Function Frenzy

Pitfalls of HCL’s try Function

In Conclusion

Next-level IaC: How Pulumi Supports Your API Economy Strategy

What is API Economy?

What is Pulumi Automation API in a nutshell?

How to build your own Service API with Pulumi Automation API?

Next Steps

Conclusion

Next Steps

Pulumi Cloud

Next-level IaC: Powering Up Pulumi with Package Ecosystems

Use Case: Determining S3 MIME types on-the-fly

The possibilities are endless

Next Steps

Pulumi Cloud

Next-level IaC: Drop those wrapper scripts and let your language do that for you

Run pre-deployment tasks with Pulumi

Fetch data to generate resources dynamically

Hook the event loop to run tasks after deployment

Wrapping up

Next Steps

Pulumi Cloud

`for_each` Loop With Conditional Expression

`for_each` Loop With Multi-Level Nested Dynamic Blocks

Execute Programming Code With `null_resource`, `local_file` and `provisioners`

If Statements Constructed With `count`

Pitfalls of HCL’s `try` Function