Pulumi Blog: Aaron Friel

Deploy Cloud Infrastructure in 30 Seconds with Pulumi AI

Luke Hoban — Wed, 20 Dec 2023 15:00:00 -0800

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

There are new intelligent cloud management capabilities available in Pulumi Copilot. Learn More

Earlier this year we launched Pulumi Neo, a purpose-built AI assistant that can create Infrastructure as Code (IaC) from natural language prompts using Pulumi. Since launch, we’ve seen incredible adoption of Pulumi AI, with over 200,000 questions asked so far and growing fast. Pulumi AI is popular with users new to Pulumi and/or new to the Cloud, but also heavily used by many of the most advanced IaC users and organizations to quickly discover solutions to new problems they need to solve. Over the last few months, we’ve driven major improvements to Pulumi AI through the recently launched Pulumi AI Answers pages with thousands of AI generated answers to common questions, improvements to code generation correctness and performance, and expansion of the languages supported by Pulumi AI.

Today, we are taking the next big step, introducing support for deploying cloud infrastructure directly from Pulumi AI. Going from idea to running cloud infrastructure is just a natural language prompt away!

This is accomplished through two new features:

The pulumi new command now offers the option to provide a Pulumi AI prompt instead of selecting a template. It then generates a template from your prompt, and lets you iterate on this before choosing to create a project and deploy it to the cloud.
Every Pulumi AI and Pulumi AI Answers page now has a Deploy with Pulumi button on it, which lets you click through a few prompts to deploy the given infrastructure into your cloud account using Pulumi Deployments and Pulumi ESC in Pulumi Cloud or the pulumi new command.

Check out Pulumi Neo or pulumi new in the CLI today.

Hope Pulumi AI Works with `pulumi new`

The pulumi new command is the starting point for almost every new Pulumi project today, offering users a choice from hundreds of templates across various cloud providers, languages and architectures.

Now, pulumi new users working with the Pulumi Cloud backend have the additional option to use Pulumi AI to generate a template on demand instead of selecting from one of the pre-defined templates.

The Pulumi AI response is streamed back directly into the CLI, and if Pulumi AI didn’t get it quite right, you can request to “revise” the question to ask Pulumi AI to adjust it’s answer with a follow-up prompt. When the results look good, you can hit “yes” to deploy the template, generating the source code in your chosen language and installing dependencies for the target cloud providers. With a simple pulumi up your new cloud infrastructure is deployed and running in the cloud.

There’s also a new –ai flag to pulumi new, so you can create a new project from an AI prompt with a command like one of the following:

pulumi new --ai "run NGINX in EC2 with a public port"
pulumi new --ai "a static website in Azure with a CDN in front"
pulumi new --ai "managed postgres database and Fargate compute for a containerized app"

Going from natural language idea to running cloud infrastructure in just seconds!

Deploying Cloud Infrastructure with Pulumi AI

Thousands of developers are already discovering answers to their questions via Pulumi AI, and we’ve been expanding on this with links to Pulumi AI from the Pulumi Registry API docs and with the archive of high quality questions and answers as part of Pulumi AI Answers. To date, users have used these answers to then copy/paste code snippets into their own Pulumi programs, presenting something of a small but meaningful barrier to entry to quickly experimenting with the resulting infrastructure.

Now, every Pulumi AI and Pulumi AI Answers page includes its own Deploy with Pulumi button. The Deploy with Pulumi button can be used along with any template or Pulumi code example to deploy that code using Pulumi Cloud via the New Project Wizard. We’ve extended that to support deploying projects defined by a Pulumi AI answer. This includes the code snippet, but also the project dependencies, a useful description, and language-specific project files.

Recently, we introduced many improvements to the New Project Wizard in Pulumi Cloud, including the ability to deploy a template with just a few clicks using Pulumi Deployments to do the deployments, the Pulumi GitHub app to template the code into a repo for you, and Pulumi ESC to provide access to short-lived credentials for your cloud environment. Once these are configured for users in an organization, any template can be deployed via the New Project Wizard and the Deploy with Pulumi button in just a few clicks.

These same features and New Project experience are now available from every Pulumi AI page. Let’s see what that looks like.

Step-by-Step Guide: Deploying with Pulumi AI

After clicking on the Deploy with Pulumi button on any Pulumi AI page, you can choose the ESC Environment to use for configuration and credentials for your environment (here I use the dev-sandbox environment which provides short-lived OIDC credentials to a development sandbox AWS account), a name for your project and stack, and the repository and folder to generate the code into.

When I click Create Project, this adds the code to the repository and kicks off a Pulumi Deployments run to deploy this new infrastructure to the cloud.

A few moments later, once that deployment completes, I see I have a Pulumi stack with the expected outputs defined by the Pulumi AI answer - the IP address of the running instance in the cloud.

And navigating there shows me my new EC2 instance serving NGINX on the Internet.

We can now continue to iterate on our infrastructure by making changes in the new Git repo folder.

With just a few clicks, we went from a natural language idea to running cloud infrastructure, checked into a Git repo, with Pulumi Deployments configured!

FAQs: Pulumi AI and Cloud Deployment

What is Pulumi AI?

Pulumi AI is an AI-powered assistant that generates Infrastructure as Code (IaC) from natural language prompts.

How does pulumi new –ai work?

The pulumi new –ai command allows you to generate and deploy cloud infrastructure using a natural language description instead of selecting a pre-defined template.

Which cloud providers does Pulumi AI support?

Pulumi AI supports AWS, Azure, Google Cloud, and other major platforms.

Can I modify the AI-generated infrastructure before deployment?

Yes! You can revise the AI-generated code before deployment to match your needs.

Where can I provide feedback on Pulumi AI?

Join the Pulumi Community Slack or join the GitHub Discussions.

Looking Ahead: The Future of Pulumi AI

Pulumi AI has already transformed how cloud engineers discover and solve their cloud infrastructure development problems. And yet, we still see incredible opportunity ahead of us. We’ve made rapid strides on improving the accuracy of AI answers, and continue to invest in further improvements. We’ve made it significantly faster to go from idea to deployed infrastructure with the pulumi new and Deploy with Pulumi button integrations, but we also see the opportunity to make these experiences even more interactive going forward. And of course, we continue to integrate Pulumi AI even more broadly throughout our product experiences, lighting up AI assistance across the cloud engineering journey.

Try it out yourself today at pulumi.com/neo!

Leveling up Pulumi AI with the Pulumi Registry

Aaron Friel — Mon, 22 May 2023 00:00:00 +0000

Pulumi AI harnesses a form of generative AI, known as large language models, to help you discover, learn, and use new cloud infrastructure APIs with ease. Think of Pulumi AI as a sophisticated compass, guiding you through the ever-changing landscape of cloud infrastructure and pointing you in the direction of the most suitable solutions for your unique requirements.

In this blog post, we’ll explore our recent enhancements to Pulumi AI, focusing on how we’ve integrated Pulumi Package schema data to generate more accurate and relevant Pulumi programs. We’ll also share examples showcasing the improvements and how they can benefit your infrastructure as code journey.

Overcoming the Challenges in Pulumi AI

As powerful as Pulumi AI may seem, it was initially limited by the training cutoff of the model. Pulumi AI wasn’t aware of new providers or changes to resources after late 2021. Additionally, the limitations in generative AI made it difficult for the model to reproduce all the input properties and relevant details of the resources it was trained on from “memory.”

These factors led to a phenomenon researchers call “hallucination.” Pulumi AI would sometimes generate an API for a non-existent provider when asked to produce a Pulumi program. To overcome these challenges, we turned to the rich Pulumi Package schema data and CrossCode technology that powers the Pulumi Registry docs, our code conversion tools, and other Pulumi enhancements.

Integrating Pulumi Package Schema Data

Think of the Pulumi Package schema as a treasure trove of information about our providers. It’s a well-organized, comprehensive library that holds the keys to unlocking the full potential of Pulumi AI. However, we faced a challenge: these schemas were much larger than any current generative AI system could handle.

What puts the “large” in large language model is the volume of data required to train and the number of parameters in the model, typically in the trillions of tokens and billions of parameters, respectively. Unfortunately, these models aren’t capable of handling that much information in their context window. The context window is the thread of conversation or the “short-term memory” of the system. It’s how one can enhance a model trained a year ago with today’s data.

To better understand the limitations, let’s talk about tokens. Instead of operating on Unicode text, these models operate on token arrays, mapping many sequences of characters (words or parts of words) to 16-bit integers. For example, “Script” is one token, and “JavaScript” is typically two tokens. We found that despite this compression of words to tokens, the latest schemas for all of our providers would use on the order of one billion tokens.

The most powerful systems to date support at most 32,000 tokens in their prompt – enough to fit a novella! That should be enough, right?

We did the math: one billion minus 32,000… And we’re still about a billion tokens over the limit.

The right information, at the right time

To work around this limitation, we need to carefully select and inject the right schema information into the context window, ensuring that the model could understand, process, and generate the best possible solutions for users.

We cracked open our college textbooks on information storage & retrieval, evaluated the latest vector databases and embedding systems, and perused the literature. One internal demo we did used a technique similar to Hypothetical Document Embeddings (HyDE). While this did improve accuracy and reduce hallucinations, that approach also requires pre-processing the user’s query. That additional step results in a much higher time-to-first-token (TTFT) latency, negatively impacting user experience.

After iterating on these approaches in the weeks after launch, we’ve implemented a method that yields comparable results to HyDE without slowing the system down. Our current approach allowed us to maintain an engaging and responsive interface while still enhancing Pulumi AI’s accuracy.

We continue to follow the latest research in this space to improve Pulumi AI, and we hope we can contribute our learnings back in this exciting field of research!

Improved User Experience and Examples

With the integration of Pulumi Package schema data, Pulumi AI now generates more accurate and relevant Pulumi programs, favoring our high-level components and the generally available providers we recommend, such as the awsx, eks, and azure-native packages.

Powerful components

For EKS clusters, Pulumi AI now typically recommends our eks provider’s high level component when asked for “an eks cluster”:

To achieve your goal of creating an EKS cluster, I recommend using the eks.Cluster Pulumi resource, which is a higher-level component encapsulating the necessary AWS and Kubernetes resources. This resource makes it easy to provision, scale, and manage EKS clusters in a Pulumi Python program.

To proceed, you’ll need to have the pulumi-eks package installed:
pip install pulumi-eks
Here’s a simple Pulumi Python program that creates an EKS cluster:
import pulumi
from pulumi_eks import Cluster

# Create an EKS cluster.
my_cluster = Cluster("my-cluster")

# Export the cluster kubeconfig.
pulumi.export("kubeconfig", my_cluster.kubeconfig)

Simple building blocks

Pulumi AI now offers a more succinct solution for common queries such as “a static website on Azure”:

To accomplish your goal, I will generate a Pulumi TypeScript program using the azure-static-website.Website resource from the Pulumi Registry. This resource is designed specifically for hosting static websites on Azure.

Here’s a Pulumi TypeScript program to deploy a static website on Azure using the azure-static-website.Website resource.
import * as pulumi from "@pulumi/pulumi";
import * as azure from "@pulumi/azure";
import * as staticWebsites from "@pulumi/azure-static-website";

const website = new staticWebsites.Website("website", {
 sitePath: websitePath,
 resourceGroupName: resourceGroup.name,
 withCDN: true,
});

New products

When asked for modern resources such as Google AlloyDB, launched six weeks ago, Pulumi AI now references Registry documentation to answer “google alloydb with an automatic backup policy”:

Based on the Pulumi Registry results and trying to accomplish your goal of creating an AlloyDB cluster with an automatic backup policy, we can create a Google Cloud Platform (GCP) AlloyDB cluster using the gcp.alloydb.Cluster resource. You can set the automatedBackupPolicy property with the necessary configuration to enable automatic backups.

Here’s a Pulumi program in TypeScript that creates an AlloyDB Cluster with an automatic backup policy:

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

const projectId = gcp.config.project;
const location = "us-central1";

const network = new gcp.compute.Network("alloydb-network", {
 autoCreateSubnetworks: false,
});

const alloydbCluster = new gcp.alloydb.Cluster("alloydbCluster", {
 clusterId: "my-alloydb-cluster",
 location,
 project: projectId,
 network: network.selfLink,
 automatedBackupPolicy: {
 enabled: true,
 labels: {
 "backup-type": "automated",
 },
 location: "us-central1",
 quantityBasedRetention: {
 count: 7,
 },
 timeBasedRetention: {
 retentionPeriod: "7d",
 },
 weeklySchedule: {
 daysOfWeeks: ["MONDAY", "WEDNESDAY", "FRIDAY"],
 startTimes: [
 {startTime: "04:00:00"},
 ],
 },
 },
});

export const alloyDBClusterName = alloydbCluster.name;

Getting Started with the Enhanced Pulumi AI

Our enhancements to Pulumi AI have made it an even more useful tool for generating accurate and relevant cloud infrastructure solutions. We’re excited about the future of Pulumi AI as we continue to integrate it into Pulumi Cloud and empower everyone to become a cloud engineer.

We’ve also added new ways to reach Pulumi AI. In the Pulumi Registry, “Explore with Pulumi AI” links provide interactive documentation. And after you’ve had a conversation with Pulumi AI, you can share it via a “Share conversation” link in the bottom right corner of the message box:

Try out Pulumi AI today, and happy coding!

Enabling Rapid Pulumi Prototyping with Rust

Aaron Friel — Wed, 21 Sep 2022 00:00:00 +0000

Pulumi enables engineers to employ the best practices of their field to infrastructure as code. The pulumi watch command is an example of this, enabling rapid prototyping and a “hot reload” style developer experience for prototyping Pulumi programs. In this post you’ll see what watch mode enables, the challenges encountered in maintaining the feature, and how we were able to use Rust to bring that feature to more of our users.

When developing a web application, hot reload frameworks enable quickly iterating and confirming the divs are centered. When implementing new features, a test runner with a watch mode makes test-driven development a breeze. In languages with an interactive shell, it’s natural to use that to experiment with new APIs. In each of these cases, engineers find value in being able to focus on writing code, not typing console commands.

That’s why Pulumi Watch Mode exists and is one of my favorite features, and it’s why I was disappointed to learn this feature was not available to users on Apple Silicon Macs. As a Windows and Linux user, I discovered this while root causing a Pulumi provider’s failing build to the library used to implement watch mode. Thus began a deep dive into cgo, file-watching libraries in Go, operating system event APIs, and finding a solution for our build issues and missing feature.

To close this feature gap for one of our most important platforms, Pulumi v3.39.0 includes a binary built with Rust to implement file-watching. pulumi watch works as it always has. And as the first component to use the Rust language, it validated Pulumi’s approach to engineering of using the best languages and tools available.

But first, what is and why use watch mode?

Watch mode monitors a project folder and automatically runs an equivalent to pulumi up with the appropriate flags to skip prompts and displays simplified output. To see how quickly this can work, I wrote a Pulumi TypeScript program to deploy an S3 bucket and bucket object “test.txt” before writing a timestamp to the same file. That allows me to measure the inner loop with pulumi watch or using an external program to run pulumi up. On each deployment, the program changes one of the files in the directory kicking off the next deployment:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as fs from 'fs/promises'

const bucket = new aws.s3.Bucket("my-bucket");

const bucketObject = new aws.s3.BucketObject("my-bucket-object", {
 bucket: bucket.id,
 key: "test.txt",
 source: new pulumi.asset.FileAsset("test.txt"),
});

bucketObject.content.apply(() => {
 fs.appendFile('test.txt', `${new Date().valueOf()}\n`);
});

On a laptop with this simple program, pulumi watch consistently completed up to 50% faster, at 2.1 seconds per iteration than when using an external tool. Watch mode was more consistent, too: the standard deviation over 50 samples was reduced to 46ms from 216ms. It does this while being easier to remember and therefore use correctly, and with more terse output tuned for the use case of quickly iterating on changes to a stack.

Watch mode has changed how I write infrastructure as code, and it brings that benefit to everyone without needing to install and learn third-party tools for file-watching.

The Challenges of File-Watching in Go

Returning to the cross-compilation issue, the team found builds failed after updating the Pulumi library dependency. Root causing this, I found that through a chain of Go package dependencies, the file-watching library became a compilation unit in provider binaries. That library had several reported issues with cross-compilation, with users reporting that it required enabling cgo and building on macOS to target macOS.

That raised two flags. First, cgo is not Go. Using cgo entails wide-ranging side effects and results in a more complex build, packaging, and support lifecycle. Second, continuous integration tools that support macOS runners are limited and typically much higher cost. It was unacceptable to allow unintentional changes in a provider or the Pulumi repository to impose those requirements on providers.

Why did the notify library have these constraints? This library depended on C library support for a macOS API called FSEvents, and its Go source files required a complex set of build tags. While this didn’t seem impossible to work around, the library remains unmaintained and would require us to maintain a fork to address these deficiencies. With no recent contributions to the library to address this or a path to removing the cgo dependency, I had to find another library.

The strongest candidate was the fsnotify library. This library supported the platforms required, did not require cgo, and supported cross-compilation. It did lack recursive directory watching, though the repository listed it as a planned feature. However, in August the maintainer archived the repository and stepped down from maintenance. Though contributors stepped up to take over ownership of the repo, it left the state of the project in doubt. Without features like recursive file watching, the team would have to maintain a fork. The search continued.

The last candidate looked at was the watcher library, which supported recursive file watching across all platforms without cgo. Was this a panacea? Unfortunately, no. It did this by falling back to the simplest method of file-watching: polling. This method works well enough that the library has not changed in four years. The downside to polling is substantially higher CPU usage and battery drain. It would also require workarounds for projects that contain large folders of dependencies, such as a node.js node_modules or python venv directory. In spiking on this, I was unsurprised to see that watching a folder with vendored dependencies was orders of magnitude slower.

The Go language ecosystem lacks libraries that meet our needs. It was possible to watch only one directory, or only support one operating system, or to be insensitive to performance concerns. I needed an option that didn’t make any of these sacrifices and, ideally, was also actively maintained.

Prototype infrastructure with watch mode

Run pulumi watch to get a hot-reload inner loop that applies changes to your stack as you save files.

Get started

From library to binary and the carcinization of Pulumi

One doesn’t have to look hard to find CLIs that tackle this problem, many of which were at the top of search results while looking for libraries. Using one of these as a plugin and reading stdout for events instead would avoid any link-time dependency for providers. With this in mind, I considered building a one-off binary using the previous utilities but ruled them out. The team would become de facto maintainers of a fork of a complex library, in addition to needing to solve the deficiencies identified above. That wasn’t an innovation token I could spend.

I evaluated the complexity of building on the watchman service, an excellent file-watching CLI open-sourced by Facebook. Unfortunately, it would be costly to maintain a wrapper with watchman as a dependency. While C++ isn’t inaccessible, it does impose a complex build system and a higher barrier to entry for engineers to contribute to it. And as both a client and server application, it wouldn’t be as simple as packaging a single binary and shelling out to it.

Finally, I spiked on a library written in Rust, using the library underpinning the watchexec CLI. All-in, this worked out to just over 100 lines of code to maintain on top of the actively maintained watchexec libraries, with a similarly small patch to the Pulumi CLI. And importantly, building the binary is a simple cargo build command that the team can run on any platform. The watchutil-rs implementation lives in GitHub and is packaged as pulumi-watch, a single-purpose binary that logs on stdout when files change.

Pulumi’s mission is to enable everyone to apply the best programming languages and tools to infrastructure as code. Implementing watch mode in Rust applied that principle to watch mode while maintaining performance, moving the feature to a well-maintained footing, and expanding support to every platform Pulumi supports.

Give pulumi watch a try with our getting started guide!