Pulumi Blog: Configuration

The Pulumi 'Push to start' GitOps Experience

Kat Morgan — Tue, 23 Jan 2024 19:41:13 +0000

As a skeptic of “quick starts” myself, I approach most marketing promises with a measure of cautious excitement. If the great and powerful algorithm, friends, or a peer brought your attention here, then I invite you to take this one seriously.

Pulumi, with its full support of many general-purpose programming languages, can appear like a chore to get started with. The feeling can haunt seasoned developers as much as practitioners new to infrastructure code.

However, I’ll show you that finding the proverbial easy street is easier than you might believe. The pulumi new developer story just gets sweeter when combined with a few other nice-to-have conveniences.

This is a “bring your own GitHub login” event, so refresh that login. You will want to grab a Pulumi Cloud login as well. With credentials in hand, you are set with everything that you need to start. Let’s go!

Hint: all of the steps in this demo are ready to run yourself from the Pulumi Devcontainer using just your web browser and a GitHub account. Even better, they are real activities that Pulumians use every day to orchestrate the cloud too!

New technology learning advice

When it comes to ice cream, sometimes just selecting your flavor can be too much. Similarly, when choosing where to start with cloud tooling the onset of “choice paralysis” can be chronic. Pulumi’s powerful toolset is both a worthy endeavor to learn and a rewarding challenge to master. There is no better time to learn than the present!

“If you are short on time and have a bike, do you stop to hop on the bike, or choose to walk because there is no time to stop and get on?”

Freezing, while great for ice cream, is not quite as satisfying when building the next cloud solution and putting it off for next week or next year can be about as useful as walking fast and pushing a perfectly good bike because there is no time to stop and get on.

How can we overcome the learning curve? Picking up and mastering a new tool can be easy if we choose to start small.

Tech learning hacks:

🏆 Do: Start
🔨 Use: Helpful Tools
🥅 Make: Achievable Goals
🎉 Win: Celebrate Successes

Recognizing achievable goals can be tricky when you are learning something new. Always over-estimate the complexity of each step. If you overestimate how complicated a task might be, the biggest risk is accomplishing the goal faster than you think! Either way, the learning journey should be enjoyable and rewarding by following this rule.

Helpful tools are more abundant than ever. I recommend Pulumi as one of the most helpful tools but that is my bias showing through. Consider how intuitive, durable, and well-integrated any tool selection you make will be. Today we’re using multiple features and services of both the Pulumi and Pulumi Cloud offering, as well as demonstrating the benefits of a tightly integrated Git, GitHub, Devcontainers, and VSCode workflow to make light work of a traditionally considerably complicated achievement.

The resulting developer stack left me feeling like I would be successful automating anything. Be sure to share your experience with us in the Pulumi Community Slack or find us on Twitter/X @PulumiCorp too!

A DevOps journey

The best solutions are often buried in the details. The only way to reach them is to dig.

In true DevOps fashion, development for this blog post featured a roller coaster of chronic scope creep, pushing deadlines, discovering details that scrapped the original plan, and during the final finish line approach too! The worry and doubt was palpable.

Have we taken on too much? Did project ambitions exceed the achievable? The solution emerged while hacking on a side quest with a friend. A single vision came into focus, tying everything together into a simple “Push to Start” Pulumi developer workflow.

In this story, it took two tries to achieve the goal. Often it can take several tries. With that backstory let’s begin this next learning adventure together.

DevOps day 0

Let’s track our DevOps journey in typical “DevOps Day” style. Because we’re counting like computers, of course, we start with zero. In other words, we need a goal, and we will be starting from scratch.

Day zero is going to be your design, planning, and prep phase. Sometimes this includes various stages of research and other times it is a matter of just getting started with hands-on investigation.

Every journey includes a beginning and an end. To demonstrate the Pulumi Push to Start developer workflow, we need a project to develop! For this adventure let’s use the Pulumi Kubernetes Provider from the Pulumi Providers registry to deploy an app on Kubernetes. This provider is one of our most popular providers, as you might guess.

In this case, we can practice with a new project from the Community Engineering team. The GitHub pulumi/devcontainer repository is a GitHub template repository containing all the fundamentals of a new Pulumi infrastructure as code GitOps project. Let’s begin with a quick tour of the GitOps repo boilerplate.

“What belongs in a GitOps git repository?”

I have seen this question raised several times during the past year. Most recently it was brought up during a great customer conversation at AWS re:Invent.

Filtering through the conflicting opinions and advice about proper GitOps repo hygiene, a few fundamentals remain consistent.

Ops Environment
- Configuration
- Dependencies
- Artifacts
Documentation
- Who it is for
- What it does
- How to Operate
- Why it is important
Actions
- Testing: Continuous Integration
- Deployment: Continuous Deployment
- Safety: Version Controls

You may have assembled projects piece by piece before. In the pulumi/devcontainer repository, we provide a turnkey “known good” starter kit which will even improve over time. As a community resource, we welcome bugs and contributions!

And now for the feature presentation, let’s get to dev’ing!

What’s in a DevOps day 0?

Day 0 usually includes aspects of planning and preparing. For this exercise, we have just a few requirements to get started. Let’s knock these out and get to the fun stuff!

Prerequisites

Now, even the fanciest push-to-start car still requires a fob, so let’s cover those bases super quick.

Web browser
GitHub account
10 minutes (approx.)

Next, let’s review the tools we will depend on for developing and deploying our project.

Commandline Utilities:

*Dependencies supplied in the Pulumi devcontainer ghcr.io/pulumi/devcontainer

Tool	Description
A Programming Language	The code part of IaC
Pulumi CLI	Infrastructure as Code Tools
Pulumi ESC	Secrets and Configuration Store
Kubectl	Kubernetes CLI
Direnv	Automated Environment Variables
KinD	Local Kubernetes Infrastructure
GitHub CLI	GitHub & Git Repository Binaries
curl, jq, act, awk, etc	Other useful tools

Cloud-native … cloud developer environment?

Besides our command line tools and the programming language of your choice, we need an editor to write our code with. Many practitioners are fluent with Vim or Emacs; others turn to editors like VSCode, IntelliJ, and others. We are going to choose the GitHub Codespaces VSCode IDE for its deep integration with GitHub, and its convenience of requiring nothing from the user other than a web browser to use.

If you are already familiar with GitHub Codespaces, then you might know what to expect next. If not, you are in for a treat, and yes that is right, we are developing for the cloud, in the cloud, with the cloud.

Alright, let’s jump in and get started by launching a new GitHub Codespaces.

👇 Click this button to begin.👇

If you prefer a more informed approach you can launch the same codespaces by navigating to the Pulumi Devcontainer Repository or you can learn more about Codespaces at this GitHub Codespaces Link.

Clicking the ‘Open in GitHub Codespaces’ button should take you directly into a new Codespaces session.

Note: This demonstration is deeply integrated into the Web based Codespaces environment offered by GitHub. Alternatively, VSCode also supports running Devcontainers with Docker locally allowing you the benefit of Pulumi Devcontainer on your own hardware.

With a new codespaces session launched you should see the VSCode IDE in your browser. If you are not already familiar with VSCode, you can learn more about it in the VSCode Docs.

Congratulations! Your developer environment is ready for some Pulumi action.

For our final Day 0 step, we still need to decide what app to deploy with our infrastructure-as-code (IaC) program.

Focusing on something fun, let’s deploy Minecraft as the app for our Kubernetes exercise. It’s a fun way to get started with Pulumi, a great way to show off your cloud skills to friends, and maybe even share with your favorite young future technologists! After all, Open Source DevOps is for everyone!

DevOps day 1

We have an environment! We have a plan! Now we need to code … or do we?

Don’t just take my word for it. The big brains of Pulumi’s AI Engineering team have delivered a brilliant Pulumi AI. Your Pulumi IaC developer sidekick makes learning and developing Pulumi just that much easier.

Q: What can Pulumi AI do? A: Today’s AI is all experimental. As an industry, we are only just beginning to discover the possibilities, but you are invited to try it yourself with the Pulumi Neo WebUI or with the pulumi new cli as demonstrated below.

`~$ pulumi new --ai`

Pulumi Neo is available via WebUI much like the ChatGPT console. Besides the familiar browser-based interface, Pulumi AI is also built into the pulumi new CLI too! Now you can prompt Pulumi AI interactively from the convenience of the command line tools you will already be using, conveniently configured in your new Codespaces IDE.

Let’s try it! (Reminder: Perform all these steps from within the Codespaces IDE.)

# First Login to Pulumi Cloud for a few features we will use in upcoming GitOps content.
pulumi login

Step 2. Create a new program

# Let's create a new pulumi with the Pulumi CLI
pulumi new

Step 3. YOLO for fun and profit!

Let’s go all in on this one and try Pulumi AI. We can always enhance the prompt with additional dialog back and forth during the pulumi new chat if it doesn’t meet our expectations right away.

It is good to keep in mind that even as advanced as AI is today, the practice is still very much in its infancy. We will save more deep dives into the differences Traditional Ops (Imperative Ops), GitOps (Automated Declarative Ops), and AI Ops (Probability Augmented GitOps) in another post, for now let’s see how close the Pulumi AI can take us.

Woohoo! We selected the AI option on CLI, let’s follow the user prompts and feed the AI a simple prompt of what we want our Pulumi project to create.

Let’s try something like this:

Write a pulumi infrastructure as code program which creates the namespce 'minecraft' and using the pulumi_kubernetes.helm.v3.Release to deploy the https://itzg.github.io/minecraft-server-charts 'minecraft-server' helm chart, sets EULA to true, deploys onto Kubernetes in the minecraft namespace."

How did we do? Let’s see what the AI came up with.

It’s looking pretty good for a simple deployment. In DevOps, nothing works till the CI testing turns green. Since fortune favors the bold let’s proceed. For my developer workflow I ran kind create cluster in the Codespaces terminal. You can do the same or add your own kubeconfig to deploy on your choice of cluster.

Of course, proceed through the remaining prompts from the pulumi new interactive CLI.

I chose Python for this project, but you can choose any of the supported languages. We can see that pulumi new creates a new Python virtual environment for us programmatically. In addition to the boilerplate supporting our new project, Pulumi AI wrote the first iteration of our Python Minecraft Deployment IaC! The next steps should be printed on the screen. Assuming your Kubernetes cluster kubeconfig is ready to roll we just have one step left.

Hint, this is where we get the name for our annual “Pulumi Up conference.

Follow the link for the full playlist of 2023 Pulumi Up sessions

pulumi up -y

DevOps day 2

Finally, it is day two. This is the beginning of a choose-your-own-adventure wonderland and the opportunities go almost as far as your imagination. This is where we commit our code and plan our next change requests and enhancements.

We can commit this code to git. Build a GitHub Actions Workflow to automate the CI testing of our IaC with every git pr or commit. Once you have a production-ready IaC, you can add another to deploy to production clusters. If you run make act you will see how comfortable testing new GitHub actions workflows can be in the convenience of your new enhanced IDE workflow. You can see the dev container workflow running yourself right away by executing make act as a starting point.

To conclude, Pulumi is not a single tool. Pulumi is a complete cloud developer ecosystem. Tools, SDKs, and cloud services come together forming the most accelerated way to create your own cloud success legends.

Conclusion

Did it deploy?

Let’s check on the status of our deployment.

# Check for your new Minecraft pod
kubectl get po

With any luck, our Pulumi Neo gave you a working program right out of the gate. For references on Kubernetes Pulumi package resources to expand your infrastructure from here head to the Pulumi Registry Kubernetes Provider page for more.

Sharing Pulumi with people is a genuine passion found among Pulumians all around the globe but we all know evidence reigns king among engineers who, like any good practitioner, expect proof. Evolving your developer workflow with Pulumi gets even better than just easy, it’s remarkably powerful and ready for you to adopt in your daily workflows right away too.

Follow PulumiTV for upcoming examples built on this solution and to see how Pulumi ESC brings safety to your secrets and more.

What’s in the recipe?

Because no hero’s journey is complete without a gruesome account of what came before, I have included many of the manual steps required to reach this state one command at a time for the ~~morbidly~~ curious as well.

click ‘details’ to expand

Git Code Repository

Now create a git code repository to version control this project infrastructure code in GitHub.

# Authenticate with GitHub before proceeding.
gh auth login --web --git-protocol https --scopes "repo,gist,read:packages,admin:org,delete_repo,codespace"

# Create new Git Repository
gh repo create workshop --public \
 --gitignore Python --license apache-2.0 \
 --description "pulumi iac developer workflow workshop"

# Initialize the git repository
git clone --recurse-submodules https://github.com/${GITHUB_USER}/workshop .

# Configure `git` cli to use the `gh` cli for authentication with GitHub
gh auth setup-git

# Configure the username and email associated with your GitHub account.
git config --global user.email ${GIT_COMMITTER_EMAIL}
git config --global user.name ${GITHUB_USER}

# List files being tracked by git and their current status.
git status

Direnv: Automatic Environment Variables

Various non-secret environment variables may be worth maintaining in code locally as well. There are many ways to do this, but here we are going to use Direnv to automatically load environment variables from a .envrc file.

# Add environment variables useful during development to .envrc
cat <<EOF >> .envrc
export NO_COLOR=true
export PULUMI_HOME=\${HOME}/.pulumi
export PULUMI_SKIP_UPDATE_CHECK=true
export PULUMI_SKIP_CONFIRMATIONS=true
export KUBECONFIG=\${PWD}/.kube/config
source venv/bin/activate 2>/dev/null || true
EOF

Now you can enable direnv on this directory.

# Enable direnv in this directory.
direnv allow

Note:

Env variables shown are for educational purposes only and should be used with care. Read more about available Pulumi Environment variables here: https://www.pulumi.com/docs/cli/environment-variables

Pulumi Devcontainer

Now, to make sure we have all of our CLI dependencies, let’s grab the Pulumi Devcontainer.

# Add the Pulumi Devcontainer git submodule
git submodule add https://github.com/pulumi/devcontainer .devcontainer
git submodule update --init --recursive .devcontainer

# Copy the devcontainer config of your choice into the top level of your project
# so it will auto select by default when you open Codespaces in the future.
cp -f .devcontainer/devcontainer.json .devcontainer.json

# Rebuild the codespaces using the newly added Pulumi Devcontainer
gh codespace rebuild --codespace ${CODESPACE_NAME}

Pulumi Cloud

Let’s login to Pulumi Cloud and initialize a new ESC environment to store our environment variables, secrets, and configuration which should be preserved and distributed securely from your Pulumi Cloud account.

# Login to Pulumi Cloud
pulumi login

# Create a new ESC Environment (optionally use another already existing environment)
pulumi env init workshop

KinD: Kubernetes-in-docker

We are going to use Kubernetes to demonstrate our Pulumi IaC. Let’s go ahead and create a new cluster now.

# Create the directory for your new kind kubeconfig
mkdir .kube

# Create a Kind Cluster
kind create cluster

# Let's make sure our kubeconfig works and pods are starting up
kubectl get pods -A

# Load Kubeconfig into Pulumi ESC as an encrypted secret for safe keeping.
# This is an example use case for storing kubeconfig which would not usually be practical
# for disposable kind kubernetes clusters like this however the approach is practical in many
# other scenarios which you may come across.
pulumi env set workshop secrets.kubeconfig.kind --secret "$(jq . -R -s < $KUBECONFIG)"
pulumi env set workshop files.KUBECONFIG --plaintext \${secrets.kubeconfig.kind}

In our environment, we maintain our secrets including API personal access tokens, kubeconfigs, and such all with Pulumi’s Environments, Secrets, and Configuration cloud service.

# Load the newly updated Pulumi ESC Environment in the local shell
eval $(pulumi env open workshop --format shell)

Now let’s create the deployment code!

If fortune favors the bold, let’s be bold on this next step and let AI write our sample code on the fly!

As this is a “hello world” style demonstration intended to showcase the developer workflow more than any specific cloud technologies, let’s deploy something fun like a Minecraft server! Be sure to watch the video demonstration of this exercise on Pulumi’s PulumiTV YouTube channel for an extra easter egg bonus step!

# Write a new Pulumi Python IaC program to deploy Minecraft on Kubernetes
pulumi new \
 --ai "Write a program using pulumi kubernetes helm v3 Release to deploy the itzg/minecraft-server helm chart on Kubernetes, and include a pulumi output to show the helm release status." \
 --description "A pulumi infrastructure as code (iac) program for deploying and serving minecraft on kubernetes" \
 --name "minecraft-on-kubernetes" \
 --language python \
 --stack "workshop" \
 --force \
 --dir .

Now let’s see if our new Infrastructure as Code Pulumi Python Codebase will deploy!

# Deploy your new Pulumi IaC on Kubernetes
pulumi stack select workshop
pulumi up

Cleanup

pulumi destroy -y --skip-preview
pulumi stack rm workshop
kind delete cluster --name kind
gh repo delete ${GITHUB_USER}/workshop

Simplify Configuration Management with Project-level Config

Fraser Waters — Wed, 02 Nov 2022 06:00:00 -0700

This blog post discusses project-level config before Pulumi ESC was released. ESC makes this and hierarchical config easy.

One of our most up-voted feature requests (with 78 thumbs ups) is to support hierarchical config. We’re happy to announce that we’ve now released the first part of plans to support this feature.

Pulumi will now allow you to set configuration values in your Pulumi.yaml file, using the given value as a default for all stacks in the project. While we expect even this first level of support will be incredibly useful to many people we also want to assure you that we have many more plans in place to make this feature even better.

How to Use Project-level Configuration

Pulumi has always had the concept of configuration per stack, and the changes for project level configuration haven’t fundamentally changed that. You can still set config for stacks via the pulumi config CLI command, or by editing the Pulumi.<stack>.yaml file. Project level config just gives you the ability to set default values for all stacks in your Pulumi.yaml.

A common request we get for this is making sure that all stacks use the same AWS region, or don’t have default providers enabled. You can now do this by setting these in your Pulumi.yaml.

name: project
runtime: go
config:
 pulumi:disable-default-providers: [“*”]
 aws:region: us-east-1

As you can see this allows setting default configuration values per-stack for configuring the engine or providers, but you can also set default values for project configuration. For example maybe you have a default count of the number of virtual machines to create:

name: project
runtime: go
config:
 vm-count: 2

The only caveat to mention about this system is that if you have object values they must be nested under a key value. So if you want to set the key obj to the object value { foo: "bar" } you would need to write it as such:

name: project
runtime: go
config:
 obj:
 value:
 foo: bar

This allows the engine to distinguish between values and schemas, which we will show below.

Defining Configuration Schemas in Pulumi

A related feature to defining values for config at the project level is being able to define a schema for the config. This allows you to define what configuration settings you expect in your Pulumi.yaml so that the engine can use that information to validate configuration for every stack.

This is done using a very similar declaration to JSON schema, or our Pulumi JSON schema (the schemas from resource providers that drive our code generators). Below is a small example of some of the things you can declare:

config:
 vm-count:
 type: integer
 description: the number of virtual machines to create
 default: 2
 subnets:
 type: array
 description: an array of subnets to create
 items:
 type: string
 vaultKey:
 type: string
 description: which vault key to use for this stack
 secret: true

These are declared in the same config block as the other project level config shown through this post. You can combine the two by declaring your project settings and defining defaults for providers together:

config:
 aws:region: us-east-2
 createVpc:
 type: boolean
 default: false

Pulumi YAML Configuration: What’s Changing?

When we released our support for simple Pulumi programs in YAML we added some support for configuration with it. This was done via the configuration key in the Pulumi.yaml file. We’ll be deprecating support for that key now, as YAML will instead use the new standard project configuration.

What’s Next for Pulumi Configuration?

This is just the start of our support for better config. We posted a long comment to GitHub earlier this year to explain some of our future plans, and we do intend to continue improving this part of the Pulumi system as described in that comment.

FAQ: Project-Level Configuration in Pulumi

What is project-level config in Pulumi?

Project-level config allows you to set default configuration values for all stacks in a Pulumi.yaml file.

Can I override project-level config for specific stacks?

Yes! You can override default values in Pulumi.<stack>.yaml or via the CLI.

How does Pulumi handle object values in config?

Objects must be nested under a value key to distinguish them from schemas.

Is the configuration key still supported in YAML?

No, it is now deprecated in favor of project-level configuration.

Where can I give feedback or ask questions?

Join the Pulumi Community Slack or join the GitHub Discussions.

Introducing the Azure Static Website Component

Sean Holung — Tue, 11 Oct 2022 00:00:00 +0000

Deploying a static website often involves provisioning a number of pieces of infrastructure and stitching those pieces together in a way to make the site accessible to your users. A static website typically consists of HTML, CSS, and JavaScript files along with any other static assets the site might need to render such as PNG and SVG files for example. These files are then uploaded to a storage bucket where they can be served from. This post will demonstrate how to easily deploy your static website to Azure and make it available for public access.

Azure Static Website Component

Pulumi components enable us to create re-usable abstractions that can perform complex tasks all behind a simple interface the user can programmatically interact with. The Azure Static Website component looks to take all of the complexity of getting a static website deployed to Azure and simplify it down to just consuming a simple component and passing it a few input arguments. This component relieves the complexities of this process by allowing you to simply tell the component where your files are located and it will take care of all the rest.

This component does the following:

Provisions an Azure storage account
Uploads the website contents located in the specified directory
Optionally can configure a CDN, custom domain, and certs.

Using the component to deploy a static website

Let’s walk through a simple example of using this component to deploy a static website. Today, we are going to provision a storage account, upload the website contents, and wire it up to an Azure CDN.

In order to follow along with this example you will need to have the following:

Pulumi CLI installed on your machine
Access to an Azure account to provision the infrastructure
Static site to upload

See our getting started guide for information on how to configure Pulumi and Azure.

We will provision a number of pieces of infrastructure in Azure to enable serving our static website. The main ones are a Storage Account to contain all of our website files and a CDN. Let’s start by creating a new Pulumi project. First, create a new directory on your machine where the Pulumi project will live. A common pattern is to include the infrastructure configuration in a directory alongside the website you will be deploying. You can create a directory called infrastructure to house the Pulumi program or you can use another name if you prefer. This allows us to easily reference the location of the contents of your site as well as keeps your infrastructure code along-side your application. Pulumi supports multiple programming languages such as TypeScript, Python, Go, .NET, and even markup languages like YAML. Once you have created the directory, run the following command for your language of choice inside the directory:

pulumi new azure-typescript

pulumi new azure-python

pulumi new azure-csharp

pulumi new azure-go

pulumi new azure-yaml

There will be some prompts that appear asking you a few questions. Feel free to accept the defaults or change them to your desired preference. This command generated a new project and created a few files for you. You can copy the code examples below for the language you chose to use.

import * as pulumi from "@pulumi/pulumi";
import * as website from "@pulumi/azure-static-website";

const site = new website.Website("site", {
 sitePath: "../site", // replace with path containing website contents
 withCDN: true,
});

export const cdnURL = site;

import pulumi
import pulumi_azure_static_website as azure_static_website

site = azure_static_website.Website("site",
 site_path="../site", # replace with path containing website contents
 with_cdn=True)
pulumi.export("cdnURL", site.cdn_url)

using System.Collections.Generic;
using Pulumi;
using AzureStaticWebsite = Pulumi.AzureStaticWebsite;

return await Deployment.RunAsync(() =>
{
 var web = new AzureStaticWebsite.Website("web", new()
 {
 WithCDN = true,
 SitePath = "../site", // replace with path containing website contents
 });

 return new Dictionary<string, object?>
 {
 ["websiteURL"] = web.CdnURL,
 };
});

package main

import (
 "github.com/pulumi/pulumi-azure-static-website/sdk/go/azure-static-website"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 web, err := azure - static - website.NewWebsite(ctx, "web", &azure-static-website.WebsiteArgs{
 WithCDN: pulumi.Bool(true),
 SitePath: pulumi.String("../site"), // replace with path containing website contents
 })
 if err != nil {
 return err
 }
 ctx.Export("websiteURL", web.CdnURL)
 return nil
 })
}

name: az-website
runtime: yaml
description: A static website build with pulumi-azure-static-website.
resources:
 web:
 type: "azure-static-website:index:Website"
 properties:
 withCDN: true
 sitePath: "../site"
outputs:
 websiteURL: ${web.cdnURL}

This code creates a new instance of the Website component and sets a couple of properties. sitePath is the location where your website contents live. The sitePath specified should be the output location of your website’s build not the overall directory where your application lives. For example, if using a framework like React, the build output is usually placed in a directory called build. In addition, we set withCDN to true to let the component know that you would like to provision a CDN as well. It also exports the cdnURL property, which is the URL where the site can be accessed. That’s all that is needed and now we are ready to deploy!

You can now run pulumi up to deploy the site to Azure. This will first show a preview of the resources that will be provisioned. You can go ahead and accept by selecting yes.

In a few minutes your site will be provisioned and ready to access. Use the cdnURL that was output at the end of the provisioning to access the site!

As you can see, the Azure static website component made this task very easy. With just a few lines of code we were able to provision our site.

If you would like to learn more about Pulumi, here are some links you may find interesting:

Visit our Getting Started to quickly get up and running with Pulumi.
View the GitHub repo for the Azure static website component.
Visit our registry page to more components we have available for you to consume.
Check out our new templates that make it quick and easy to deploy common cloud architectures.

Create an AWS Static Website Fast with Angular and Pulumi

Sean Holung — Wed, 24 Aug 2022 00:00:00 +0000

In this blog post, we’re going to use some Angular framework components to assemble a static website and then use Pulumi and its AWS Static Website component to deploy it to AWS. The website is for a café called the Pulumi Café. It will contain two pages, one an About page and the other a Menu page, as well as some navigational pieces.

To follow this example, you need to have both Angular and Pulumi installed. (Here’s a link to the Pulumi installation instructions.) You’ll also need an AWS account.

What is Angular?

Angular is an open-source UI framework for building single-page client applications with HTML and TypeScript. Angular itself is written in TypeScript and its architecture follows the Model-View-Controller (MVC) design pattern.

What are Components in Angular?

In Angular, the basic building blocks of a website are called components. These are reusable structures that contain their own logic and functionality. For example, you could create a button as an Angular component and program it to have a particular look and a particular behavior. Then, you would reuse the button component wherever you need it in the website. The button could also be configurable meaning that, based on different properties, you could change its look and behavior.

A simple website, like the one we’ll be building in this post, is mainly created from a group of Angular components.

What is the AWS Static Website Component?

The AWS Static Website component from Pulumi makes it easy to deploy an AWS S3 static website and, optionally, add a CloudFront content distribution network (CDN). While you can use any of the programming languages Pulumi supports (TypeScript, JavaScript, Python, Go, .NET, Java, and YAML), we’ll be using TypeScript, since that is what the Angular framework uses.

The great advantage of the AWS Static Website component is that you can have a complete, functioning site in a few minutes. The component encapsulates all the complex logic that an AWS static website requires. Without the component, you would need to spend hours or even days writing and debugging code to get the same result.

If you’re interested in learning more about the component, read Deploy Static Websites to AWS using 10 lines of YAML.

The Angular Components

All the Angular components you need to build the website are available in the GitHub repository. The repository contains:

A component for the left navigation bar.
A component for the top navigation bar.
A component for the Home page.
A component for the Menu page.

We also use some components that come with the Angular framework.

Each of the components has a TypeScript file that contains the programming logic and an HTML file that describes how the component will render.

Building the Pulumi Café Website

Here are the instructions to build the website.

Clone the GitHub repository located here.
Run npm install to install the website’s node dependencies.

Deploy the Site with Pulumi

You’re now ready to set up Pulumi to deploy the site to AWS.

Create a directory named pulumi in the root of the project directory and then navigate to it with this command:
```
mkdir pulumi && cd pulumi
```
You’re now in the pulumi directory. Generate a Pulumi project file and create a Pulumi.yaml file with the following command. You will be asked a few questions from the prompt. Use angular-static-website for the project name and accept the suggested defaults for the remaining questions:
```
pulumi new aws-yaml
```

Modify the .yaml File

We need to add a few lines of code to the Pulumi.yaml file. In the resources section, the code references the AWS Static Website component and sets the sitePath property to the location of the dist build directory. We also want to know the URL of the site once it’s deployed. That’s what the outputs section does. The websiteURL parameter is one of the outputs that the AWS Static Website component provides. Replace the contents of Pulumi.yaml with the following code:

name: angular-static-website
runtime: yaml
description: Example of deploying Angular with Pulumi
resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "../dist/"
outputs:
 websiteURL: ${web.websiteURL}

The AWS Static Website component is the reason you only need a few lines of code to deploy the site. By using it, all the work of setting up AWS is done for you.

To deploy the website, type pulumi up. You’ll first see a preview of all the resources that will be created. Confirm that you want to proceed and Pulumi deploys the website to AWS.

It’s that simple. With just a few lines of code and one command you deployed your site. Navigate to the site and you’ll see the Home page for the café.

You can see the navigation components on the left-hand side. Click Menu to see the Menu page.

Manage Your Site with Pulumi

Along with making it simple to deploy your site, Pulumi also makes it easy to perform some management tasks. For example, If you want to take down the site, run pulumi destroy. You never have to worry that you’ll forget to remove some resource and continue to pay for it. Pulumi also keeps track of changes to website files, can generate resource graphs and give you complete information about the resources you’re using.

Learn More

If you want to learn more about Pulumi and the AWS Static Website component, here are a few links you might find interesting:

Along with the Deploy Static Websites to AWS using 10 lines of YAML blog post, take a look at the AWS Static Website reference page.
If you want to learn about other Pulumi components, the registry page catalogs them all.
If you want to view more examples of how to use Pulumi, go to the Pulumi examples repo.
If you want to delve deeper into Pulumi, the Getting Started guide is a good first step.
If you’re interested in learning more about Pulumi concepts, try the Concepts page.
If you want to learn more about Angular, see their Getting Started docs.

Deploy Static Websites to AWS using 10 lines of YAML

Sean Holung — Thu, 07 Jul 2022 00:00:00 +0000

The AWS Static Website component makes it easy to deploy an AWS S3 static website and, optionally, add a CloudFront content distribution network (CDN). While you can use any of the programming languages Pulumi supports (TypeScript, JavaScript, Python, Go, .NET, Java, and YAML), the component is particularly useful if you use YAML or JSON. With the AWS Static Website component, you’ll have a complete, functioning site in a few minutes. Without it, you can spend hours or even days to get the same result.

The component is designed to be simple to use and accessible to a broad range of developers. The only prerequisites are a basic knowledge of YAML or JSON, an AWS account, and an understanding of what a static website is. Even if you’re not used to thinking about infrastructure as code (IaC), you can use the component and let it handle the complexities that make setting up a website so time consuming.

You will need to have Pulumi on your machine to use the component because you run the pulumi up command to deploy the website. There’s a link to the installation instructions at the end of this article. You’ll also need an AWS account.

What the Component Does

The component deploys a static website to AWS and, optionally, a CloudFront CDN. You simply add a few lines of code, set a few configuration parameters, run pulumi up and let the component do the rest. The component sets up:

An Amazon S3 bucket
An Amazon CloudFront CDN (optional)
- An Amazon Route 53 DNS record
- An Amazon Certificate Manager SSL/TLS certificate

Some Examples

Let’s look at two examples of how to use the component. We’re going to use YAML in this post, but JSON is also supported as well.

Create a Website

This YAML code creates a basic static website.

name: react-static-website
runtime: yaml
description: Example of deploying React with Pulumi
resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "../my-website/build"
outputs:
 websiteURL: ${web.websiteURL}

This is the JSON equivalent of the code shown above.

{
 "name": "react-static-website",
 "runtime": "yaml",
 "description": "Example of deploying React with Pulumi",
 "resources": {
 "web": {
 "type": "aws-static-website:index:Website",
 "properties": {
 "sitePath": "../my-website/build"
 }
 }
 },
 "outputs": {
 "websiteURL": "${web.websiteURL}"
 }
}

This example code deploys a static React website but you can use the component with any other static website framework, such as Hugo. The only parameter that would change is the output directory specified in the sitePath property.

The first two lines of this example give the project a name and tell Pulumi that the code is YAML. The next line is simply a description of the project.

Next you allocate a resource. This is a Pulumi term. Resources are the fundamental units that make up your cloud infrastructure and Pulumi provides many resources that are ready for you to use. Here, you only need one: aws-static-website:Website, which is the AWS Static Website component. The resource has an input property, sitePath, which is the root directory that contains the website’s contents. Of course, you can use any path you want.

Finally, you’ll probably want to know the URL of your new site once it’s deployed. That’s what the outputs line does. websiteURL is one of the outputs that the component provides.

Get Started

To get started, go to the folder that contains your static website project. Then, follow these steps.

Create a directory named infrastructure alongside your website project directory with this command:
```
mkdir infrastructure && cd infrastructure
```
You’re now in the infrastructure directory. Generate a Pulumi.yaml file with this command:
```
pulumi new aws-yaml
```
Here’s an example of what the final directory structure looks like.
```
| react-project-dir
├── my-website
| ├── build
│ ├── index.html
│ ├── main.css
├── infrastructure
│ ├── Pulumi.yaml
```
Note: Pulumi will also add some other files to the infrastructure directory, such as a file that contains information specific to your Pulumi stack.

Copy the resources section and the outputs section from the example and add it to the end of the Pulumi.yaml file.

resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "../my-website/build"
outputs:
 websiteURL: ${web.websiteURL}

Deploy the Website

To deploy the website, type pulumi up. You’ll first see a preview of all the resources that will be created. You then confirm that you want to proceed and Pulumi deploys the website.

What You Gain

The entire example has 10 lines of YAML code. With this, you will get an S3 bucket provisioned with all of your website’s contents. If you looked at the preview screen when you ran pulumi up, you saw the many resources that actually had to be deployed to create the site. You never see any of that complexity. Here’s an example of what’s happening under the hood.

If you follow the View Live link, you will go to the Pulumi Service, where you can see all of the information available about a deployment and explore the resources that were deployed. For example, you can see a resource graph and a list of resources. Here’s an example of a resource graph.

After Pulumi deploys the website, it will display any outputs you specified which, in our example, is the URL of the site. as well as the number of resources created.

Input Properties

The AWS Static Website component has several input properties. Here are three of the most commonly used.

sitePath - the root directory that contains the website’s contents (required).
withCDN - sets a flag to provision a CloudFront CDN.
targetDomain - the domain that serves the content. A Route53 hosted zone must exist for this domain if you specify this option.

See the component’s reference page for the other properties this component supports.

Output Properties

The AWS Static Website component has several output properties. The one you’ll probably use the most is websiteURL, which is the URL to access the website. See the component’s reference page for the others.

Add a CloudFront CDN

To add a CloudFront CDN, you only need to add one more line of code.

name: react-static-website
runtime: yaml
description: Example of deploying React with Pulumi
resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "./my-website/build"
 withCDN: true
outputs:
 websiteURL: ${web.websiteURL}

All you have to do is add the line withCDN: true and the component sets up the CDN instance for you. Again, run pulumi up, confirm that you want to proceed, and Pulumi deploys the website. Here’s an example of what you’ll see.

And here’s the output.

More Pulumi Advantages

We’ve already seen how much Pulumi simplifies creating a website. Pulumi also keeps track of any changes you make to any of the website files. It makes those changes the next time you run pulumi up. Here’s an example screenshot. The changes are marked in the Plan column.

If you simply want to take an inventory of what’s changed, you can run pulumi up and not proceed with the confirmation. Alternatively, you can run pulumi preview.

If you want to take down the site, run pulumi destroy. Pulumi marks all the resources for deletion. It first previews the changes and, if you confirm, removes them from your AWS account. You don’t have to worry that you’ll forget to remove some resource and continue to pay for it. Here’s an example of what you’d see if you ran pulumi destroy on your AWS S3 static website.

Learn More

If you’re curious and want to learn more about Pulumi and the AWS Static Website component, here are a few links you might find interesting:

You can examine the GitHub repository that contains the code for the component.
If you want to quickly try out the component, this GitHub repo houses a complete React example project that you can clone.
If you want to learn about other Pulumi components, take a look at the registry page, which catalogs them all.
If you want to view more examples of how to use Pulumi, take a look at the Pulumi examples repo.
If you want to delve deeper into Pulumi, the Getting Started guide is a good first step.
If you’re interested in learning more about Pulumi concepts, try the Concepts page.

Install Pulumi

To install Pulumi for AWS, go to Before You Begin. There are instructions for macOS, Windows and Linux. You need to have an AWS account.

Shared configuration stacks with AWS Systems Manager

Christian Nunciato — Thu, 30 Jun 2022 00:00:00 +0000

One thing I love about Pulumi is how easy it is to configure a stack. As a builder mainly of web applications, I’m always thinking about how I’ll configure my apps from one environment to the next, and being able to use Pulumi’s built-in support for configuration and secrets to manage the API keys and database credentials for my dev, staging, and production stacks individually is incredibly convenient.

For larger teams and organizations, though, where multiple applications rely on a set of common configuration settings — dozens of apps, say, depending on the same API service or database — having to keep all of those config settings in sync across all of those individually can become a bit of a pain. When this happens, you may find yourself looking for ways to extract those settings into some sort of a service to allow you to manage them easily in one place, and in a way that allows any application to inherit them automatically.

There are lots of ways of addressing this sort of problem. One that I like is AWS Systems Manager (SSM) — specifically, SSM Parameter Store. With Systems Manager and Parameter Store, you can store plain-text and encrypted secrets and expose them easily to any other resource in your AWS infrastructure. Parameter Store is a powerful tool to have in your cloud-programming toolbox — and combined with Pulumi, and in particular Pulumi stack references, you can use it to build specialized stacks make managing shared configuration simple.

In this post, I’ll show you one way to do this. We’ll start with a simple Pulumi stack that defines a few config settings with Parameter Store, then stand up a couple of other stacks to illustrate how to inherit and use those settings. We’ll also use Pulumi YAML, one of the newest additions to the Pulumi language family, to give you an opportunity to kick the tires on that as well.

Let’s get going.

Sketching it out

The plan is to begin with a Pulumi stack to act as the keeper of our hypothetical organization’s shared configuration. This stack will define two config settings: a message of the day, which we’ll store in plain text, and a secret message of the day (ssh!), which we’ll store as an encrypted secret. Later, we’ll build two stacks to act as downstream consumers of this shared-config stack — one, a static website that’ll use our config values to render some text in the browser, the other a serverless function that’ll return those values as JSON. Both will use Pulumi stack references to obtain their config values from Parameter Store. Here’s a visual:

Let’s start Stack 1 — the shared-config stack.

Building the shared config stack

First, make sure you’ve installed and configured Pulumi for AWS, and then, in your shell of choice, create a new folder to house our three stacks-to-be:

$ mkdir shared-config-with-ssm && cd shared-config-with-ssm
$ mkdir shared-config my-website my-service

This’ll leave you with three folders to work with, one for each stack:

$ tree .
.
├── my-service
├── my-website
└── shared-config

Change to the shared-config folder and generate a new Pulumi YAML project, accepting the defaults to create a new dev stack:

$ cd shared-config
$ pulumi new aws-yaml

When the new-project wizard completes, add two new settings for the dev stack with pulumi config set:

$ pulumi config set motd 'Hello from Pulumi!'
$ pulumi config set motd_secret 'Ssh! This is a secret.' --secret

These are the two values we’re going to make available to Stacks 2 and 3.

Now let’s build out the Pulumi program to provision these items as parameters in AWS Systems Manager. Replace the contents of Pulumi.yaml with the following YAML program, which reads the config values we just set, declares two new aws.ssm.Parameter resources (using fully-qualified paths as a best practice), and exports their parameter names as Pulumi stack outputs:

name: shared-config
runtime: yaml

# The values we're reading from stack configuration.
configuration:
 motd:
 type: String
 motd_secret:
 type: String

# The AWS SSM Parameters managed by this stack.
resources:
 motd_param:
 type: aws:ssm:Parameter
 properties:
 name: /shared-config/dev/motd
 value: ${motd}
 type: String
 motd_secret_param:
 type: aws:ssm:Parameter
 properties:
 name: /shared-config/dev/motd_secret
 value: ${motd_secret}
 type: SecureString

# Outputs are the fully-qualified AWS SSM Parameter names
# that consuming stacks can use to retrieve associated values.
outputs:
 motd_param_name: ${motd_param.name}
 motd_secret_param_name: ${motd_secret_param.name}

At this point, you might be wondering why we’re only exporting our parameters’ names, and not their values.

We certainly could export their values, since we already have them at hand. But if we did, there wouldn’t be much point to using Parameter Store at all, since Pulumi itself is perfectly capable of managing plain-text and encrypted values, and stack references give other stacks an easy way of accessing them programmatically. Why not export the values, then?

For one, it’d be less flexible. If we exposed these values as stack outputs, downstream stacks referencing them would only be able to read them at deploy-time — i.e., during their own Pulumi updates. For shared-config values that change rarely, that might be okay, but for others, it’d be less than ideal in that all downstream stacks would have to be updated immediately whenever one of those values were changed.

Secondly, it keeps secrets a bit more secret. Pulumi is great at protecting the secrets it manages, even across stack boundaries like these, but applications still need a way to get access to the underlying plain-text values of secrets at runtime. Lambda functions, for example, need access tokens and database passwords to do the work we need them to do — but figuring out how to provide them with those raw values safely can be tricky. By requiring client applications to fetch secrets from Parameter Store, we can keep their plain-text values out of the bodies of our Lambda functions and prevent them from surfacing inadvertently in the AWS Lambda Console.

Finally, it leaves room for the occasional out-of-band change. Now and then, you might be faced with having to update a value in Parameter Store directly — e.g., with the AWS Console or CLI. Downstream stacks not reading from Parameter Store, however, wouldn’t have access to these updated values until after the shared-config stack were refreshed and redeployed — and would themselves have to be redeployed in order to use them.

Perhaps that was more explanation than necessary for what amounts to just a few lines of YAML, but hopefully it gives you a sense of the kinds of things to be thinking about as you explore solutions like this on your own. For now, let’s forge ahead and get this first stack deployed.

Back in your terminal, do that with pulumi up. The deployment should take only a few seconds to complete:

$ pulumi up
...

Updating (dev)

 Type Name Status
 + pulumi:pulumi:Stack shared-config-dev created
 + ├─ aws:ssm:Parameter motd_param created
 + └─ aws:ssm:Parameter motd_secret_param created

Outputs:
 motd_param_name : "/shared-config/dev/motd"
 motd_secret_param_name: "/shared-config/dev/motd_secret"

Resources:
 + 3 created

Duration: 4s

Notice we’ve named our outputs clearly, and that the secret message is safely encrypted in Pulumi.dev.yaml and in Parameter Store. You can confirm the latter by visiting the AWS Console:

Now let’s put these two new values to practical use.

Fetching configuration at deploy-time

Assuming you’re still in the shared-config folder, change to the my-website folder you created a few moments ago and generate another YAML project, this one for our humble little website:

$ cd ../my-website
$ pulumi new aws-yaml

Once again, follow the prompts to create a new dev stack, then replace the contents of Pulumi.yaml with the program below. Here, we’re fetching the names of the parameters being managed by shared-config, pulling their values from Systems Manager, and creating a one-page website to render those values in the browser. (Be sure to adjust the pulumi:pulumi:StackReferences resource to point to your stack instead of mine.)

name: my-website
runtime: yaml

variables:

 # Get the names of the parameters we care about from the shared-config stack.
 motd_param_ref: ${my-stack-reference.outputs["motd_param_name"]}
 motd_secret_param_ref: ${my-stack-reference.outputs["motd_secret_param_name"]}

 # Fetch (and decrypt) their values from Systems Manager.
 motd_param:
 Fn::Invoke:
 Function: aws:ssm:getParameter
 Arguments:
 name: ${motd_param_ref}
 motd_secret_param:
 Fn::Invoke:
 Function: aws:ssm:getParameter
 Arguments:
 name: ${motd_secret_param_ref}
 withDecryption: true

resources:
 # Create the stack reference
 my-stack-reference:
 type: pulumi:pulumi:StackReference
 properties:
 name: cnunciato/shared-config/dev  # <-- Change this.

 # Create an S3 bucket and configure it as a website.
 my-bucket:
 type: aws:s3:Bucket
 properties:
 website:
 indexDocument: index.html

 # Create a homepage and render the values.
 index.html:
 type: aws:s3:BucketObject
 properties:
 bucket: ${my-bucket}
 acl: public-read
 contentType: text/html
 content: |
 <html>
 <head>
 <meta charset="utf-8">
 <style>
 span { display: none; }
 button:active + span { display: inline; }
 </style>
 </head>
 <body>
 <h1>${motd_param.value}</h1>
 <button>Reveal the secret message!</button>
 <span>${motd_secret_param.value}</span>
 </body>
 </html>

outputs:

 # Export the publicly accessible URL of the website.
 url: http://${my-bucket.websiteEndpoint}

Save the file and deploy the stack, which again should take only a few seconds to complete:

$ pulumi up
...

Updating (dev)

 Type Name Status
 + pulumi:pulumi:Stack my-website-dev created
 + ├─ aws:s3:Bucket my-bucket created
 + └─ aws:s3:BucketObject index.html created

Outputs:
 url: "http://my-bucket-269e711.s3-website-us-west-2.amazonaws.com"

Resources:
 + 3 created

Duration: 5s

With the exported URL, open the website in your favorite browser, and you should also see the message of the day on the homepage, along with a button you can click to reveal the secret message:

If you’re using the Pulumi Service, which tracks dependencies between stacks, you should see that the my-website stack is now listed as a downstream consumer of shared-config:

Now, when the hypothetical team managing the shared-config stack decides it’s time to update the message of the day, it can do so with a pulumi config set and pulumi up, and my-website will be able to pick up the new message automatically.

However, it’s worth mentioning that that won’t happen immediately, as the my-website stack is configured to pull the message from Systems Manager at deploy-time. If we want to reflect the new value on the homepage, we’ll either need to deploy the my-website again or come up with a way to fetch the message at runtime — which brings us to our third and final stack, in which you’ll learn how to do just that.

Fetching configuration at runtime

The easiest way to see this in action is with AWS Lambda. With Lambda and Pulumi’s support for function serialization, we can capture the parameter names with stack references, pass them into the Lambda at deploy-time (as plain-text strings, which is fine, because they’re just names), and fetch their values from Systems Manager at runtime — i.e., when the Lambda is invoked sometime later — with the AWS SDK for Node.js. This way, any updates made by the shared-config team will be usable by our Lambda function immediately, no redeployment necessary.

So to finish things off, change to the my-service folder you created earlier and generates a third and final project, this one with TypeScript, again accepting the defaults to create a dev stack for it:

$ cd ../my-service
$ pulumi new aws-typescript

Replace the contents of index.ts with the following program. As before, we’re using StackReferences to read our parameter names from the shared-config stack — except here, because StackRerefences are provided as pulumi.Outputs, which can’t be serialized into the Lambda, we’ll need to use .get() to unwrap the parameter name into a string so we can use it at runtime with the AWS SDK:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Get the names of the parameters we care about from the shared-config stack.
const stack = new pulumi.StackReference("cnunciato/shared-config/dev"); // <-- Your name here.
const motdParamRef = stack.requireOutput("motd_param_name");
const motdSecretParamRef = stack.requireOutput("motd_secret_param_name");

const callback = new aws.lambda.CallbackFunction("lambda", {
 policies: [
 aws.iam.ManagedPolicy.LambdaFullAccess,

 // Give the Lambda read-only access to AWS Systems Manager.
 aws.iam.ManagedPolicy.AmazonSSMReadOnlyAccess
 ],
 callback: async () => {

 // Use the AWS SDK for JavaScript to fetch parameter values from Systems Manager.
 const ssm = new aws.sdk.SSM();

 // Fetch the message of the day.
 const motdParam = await ssm.getParameter({
 Name: motdParamRef.get(), // <-- Use .get() for you runtime access to Output values.
 }).promise();

 // Fetch the secret message of the day.
 const motdSecretParam = await ssm.getParameter({
 Name: motdSecretParamRef.get(),
 WithDecryption: true,
 }).promise();

 return {
 statusCode: 200,
 contentType: "application/json",

 // Return the result.
 body: JSON.stringify({
 motd: motdParam.Parameter?.Value,
 motd_secret: motdSecretParam.Parameter?.Value
 }),
 };
 },
});

const lambdaUrl = new aws.lambda.FunctionUrl("lambda-url", {
 functionName: callback.name,
 authorizationType: "NONE",
});

// Export the publicly accessible URL of the service.
export const url = lambdaUrl.functionUrl;

When you’re ready, deploy the my-service stack with pulumi up:

$ pulumi up
...

Updating (dev)

 Type Name Status
 + pulumi:pulumi:Stack my-service-dev created
 + ├─ aws:iam:Role lambda created
 + ├─ aws:lambda:Function lambda created
 + ├─ aws:iam:RolePolicyAttachment lambda-b63d666c created
 + ├─ aws:iam:RolePolicyAttachment lambda-b5aeb6b6 created
 + └─ aws:lambda:FunctionUrl lambda-url created

Outputs:
 url: "https://6y3vkpktyybylhua266bu7yplu0dgqgv.lambda-url.us-west-2.on.aws/"

Resources:
 + 6 created

Duration: 19s

Verify the deployment with a request to the Lambda function’s public endpoint:

$ curl $(pulumi stack output url)
{
 "motd": "Hello from Pulumi!",
 "motd_secret": "Ssh! This is a secret."
}

That tells you the runtime lookup is working, which is great.

Now try updating the message of the day. Change to the shared-config folder, use pulumi config set to update the stack’s configured value, and finally, run pulumi up to write the new value to Systems Manager:

$ cd ../shared-config
$ pulumi config set motd 'Hello *again* from Pulumi!'
$ pulumi up
...

Updating (dev)

 Type Name Plan Info
 pulumi:pulumi:Stack shared-config-dev
 ~ └─ aws:ssm:Parameter motd_param updated [diff: ~value]

Resources:
 ~ 1 updated
 2 unchanged

If all goes well, you should be able to invoke the Lambda again and see the new value immediately:

$ cd ../my-service
$ curl $(pulumi stack output url)
{
 "motd": "Hello *again* from Pulumi!",
 "motd_secret": "Ssh! This is a secret."
}

And with that, we’re done. You can tear everything down with pulumi destroy:

$ cd ..
$ pulumi -C my-service destroy
$ pulumi -C my-website destroy
$ pulumi -C shared-config destroy

Wrapping up, and next steps

Managing configuration at scale is a much bigger topic than we could possibly cover in a blog post, but hopefully you’ve learned enough here to fuel some experimentation of your own. You’ll find the full source code for this walkthrough on GitHub as well.

Additionally, as an alternative to Parameter Store, you might also want to check out AWS Secrets Manager. It’s built specifically for managing secrets, and gives you additional capabilities like rotation and auditing, so depending on your use case, it could be a better fit for certain scenarios.

Happy configuring!

Deploy WordPress to AWS using Pulumi and Ansible

Joe Duffy — Mon, 27 Jun 2022 08:00:00 -0700

There are two primary kinds of infrastructure as code tools: configuration management, like Ansible, Chef, and Puppet, which configure, patch, or upgrade existing servers, and provisioning, like Pulumi, Terraform, and CloudFormation, which create, update, and delete the underlying infrastructure itself. Provisioning has taken over from configuration management as the dominant form of IaC over the past decade as we’ve shifted to modern immutable infrastructure architectures that use containers, serverless, and managed services. And yet, configuration management still remains important and relevant, especially for heritage, stateful, server-centric, and on-prem or hybrid solutions. The good news is that it doesn’t need to be either-or choice: the two approaches are complementary. In this post, you’ll see how and why you might combine them by deploying a WordPress Server to AWS by provisioning infrastructure with Pulumi and configuring the server with Ansible.

Preamble: On Provisioning and Configuration

Before diving in, why might you want to provisioning and configuration IaC tools together?

Provisioning is more common these days because modern workloads using containers, managed services, and serverless architectures are amenable to immutable infrastructure approaches — replacing instead of patching. This entails fewer state transitions and failure modes to consider, making automation less error prone, more repeatable, and more scalable. This streamlines deploying infrastructure from CI/CD or other advanced automation, scaling up to 10s, 100s, or even 1,000s of environments repeatably.

This distinction is sometimes referred to as the difference between “cattle” and “pets”. In this analogy, cattle are given numbers like vm0042.cern.ch, are almost identical to all other cattle, and when they get ill, you get another one. In contrast, pets are given names like myspecialpet.cern.ch, are unique and lovingly hand-raised and cared for, and when they get ill, you nurse them back to health.

To illustrate what these differences mean from an infrastructure as code perspective, let’s say you have a server running an old version of WordPress and want to upgrade to the latest:

With configuration management, it is a pet, so you would patch the server, upgrading it from version X to the new version Y. This is challenging because it requires knowing all possible X=>Y transitions and potential failure modes. Imagine the upgrade from X=>Y fails — how do you recover? Is it a manual recovery? And imagine version Z comes out: is upgrading X=>Z the same upgrade path as Y=>Z? Or is each permutation different? It becomes very difficult to predict and carefully consider all combinations of upgrade paths and failures.
With provisioning, it is cattle, so you don’t worry about such things. You’d place your server, possibly many of them, behind a stable IP address and/or load balancer and, instead of patching, you would provision one or more new servers running version Y (or Z), redirect the IP address and/or load balancer to the new server(s), and then destroy the old server(s) running the old version X after traffic has been redirected. This reduces the space of state transitions to consider and eliminates a whole host of failure recovery scenarios. If something fails, you just fix the problem, and rinse and repeat; provisioning tools are better at picking up where they left off, even from partial states. In fact, this workflow is precisely the way most container-based systems, including Kubernetes, work, generalizing common patterns.

Many heritage workloads still use configuration management tools for mutable infrastructure like virtual machine (VM) servers running in on-prem or hybrid environments. Provisioning can work too, however, many VM-centric tools and practices formed over the years around mutable infrastructure. In some cases with stateful workloads it may legitimitely be preferable to continue using VMs and configuration management. And even if you’re not choosing it for new workloads, configuration management is definitely here to stay — many organizations have mission-critical workloads and procedures built up around this family of tools.

For this post, imagine your organization decided to stick with Ansible configuration management for managing a stateful WordPress server, but is otherwise modernizing by using Pulumi IaC to provision AWS infrastructure.

Pulumi and Ansible Better Together - The Approach

Our goal is to set up WordPress in AWS in an automated and repeatable manner. It should be easy to scale a single environment based on demand, upgrade the infrastructure as requirements evolve, and repeatably set up and tear down entirely new environments. These are all areas where IaC shines.

Here’s a quick sketch of the approach:

Create a Pulumi program in your language of choice (Python, YAML, any Node.js language (such as JavaScript, TypeScript), Go, any .NET language (such as C#), or any JVM language (such as Java, Groovy, Scala, or Kotlin). This program defines the declarative infrastructure specification.
Declare the AWS infrastructure topology using Pulumi’s native programming model. This infrastructure includes a Virtual Private Cloud (VPC), public and private subnets, security groups, a MySQL RDS database that WordPress will store data in, and an EC2 instance that’s accessible over the Internet to run the WordPress server, among a few other supporting bits of infrastructure.

Here is a visual diagram of this AWS architecture:
Declare configuration commands, still all within Pulumi’s programming model, to perform two tasks:
1. Update packages over SSH, using yum to ensure prerequisites like Python are present.
2. Run the Ansible Playbook on the machine running Pulumi as the Ansible control node. This installs and configures WordPress on the new infrastructure.
Run a single command, pulumi up, to orchestrate everything, including provisioning or updating the AWS infrastructure and running commands and Ansible playbooks to perform the on-VM configuration.
Finally, the WordPress server’s IP address is printed, and our server is ready to go!

Now that you’ve got an idea of the overall game plan, let’s get started — first, you’ll create a Pulumi project to house your IaC.

Creating Your Infrastructure as Code Project

To start, you’ll create a Pulumi IaC project, which will declare all of the supporting AWS infrastructure into which you’ll then install WordPress.

Note: If you’re following along, be sure to install the prerequisites:

Install Pulumi

Install Ansible

Configure AWS Credentials

First, create a new Pulumi project in your language of choice:

$ pulumi new aws-typescript

$ pulumi new aws-python

$ pulumi new aws-go

$ pulumi new aws-csharp

$ pulumi new aws-yaml

Next, to make our infrastructure setup easy to vary between environments, the code leverages Pulumi’s “configuration” system — not to be confused with configuration management tools — which just allows you to vary provisioning settings for each stack in case your environments have different needs. For instance, perhaps you have different SSH keys, or want larger instance sizes, in development versus production:

const config = new pulumi.Config();
// A path to the EC2 keypair's public key:
const publicKeyPath = config.require("publicKeyPath");
// A path to the EC2 keypair's private key:
const privateKeyPath = config.require("privateKeyPath");
// The WordPress database size:
const dbInstanceSize = config.get("dbInstanceSize") || "db.t3.small";
// The WordPress database name:
const dbName = config.get("dbName") || "wordpressdb";
// The WordPress database user's name:
const dbUsername = config.get("dbUsername") || "admin";
// The WordPress database user's password:
const dbPassword = config.requireSecret("dbPassword");
// The WordPress EC2 instance's size:
const ec2InstanceSize = config.get("ec2InstanceSize") || "t3.small";

config = pulumi.Config()
# A path to the EC2 keypair's public key:
public_key_path = config.require("publicKeyPath")
# A path to the EC2 keypair's private key:
private_key_path = config.require("privateKeyPath")
# The WordPress database size:
db_instance_size = config.get("dbInstanceSize") or "db.t3.small"
# The WordPress database name:
db_name = config.get("dbName") or "wordpressdb"
# The WordPress database user's name:
db_username = config.get("dbUsername") or "admin"
# The WordPress database user's password:
db_password = config.require_secret("dbPassword")
# The WordPress EC2 instance's size:
ec2_instance_size = config.get("ec2InstanceSize") or "t3.small"

cfg := config.New(ctx, "")
// A path to the EC2 keypair's public key:
publicKeyPath := cfg.Require("publicKeyPath")
// A path to the EC2 keypair's private key:
privateKeyPath := cfg.Require("privateKeyPath")
// The WordPress database size:
dbInstanceSize := cfg.Get("dbInstanceSize")
if dbInstanceSize == "" {
 dbInstanceSize = "db.t3.small"
}
// The WordPress database name:
dbName := cfg.Get("dbName")
if dbName == "" {
 dbName = "wordpressdb"
}
// The WordPress database user's name:
dbUsername := cfg.Get("dbUsername")
if dbUsername == "" {
 dbUsername = "admin"
}
// The WordPress database user's password:
dbPassword := cfg.RequireSecret("dbPassword")
// The WordPress EC2 instance's size:
ec2InstanceSize := cfg.Get("ec2InstanceSize")
if ec2InstanceSize == "" {
 ec2InstanceSize = "t3.small"
}

var config = new Config();
// A path to the EC2 keypair's public key:
var publicKeyPath = config.Require("publicKeyPath");
// A path to the EC2 keypair's private key:
var privateKeyPath = config.Require("privateKeyPath");
// The WordPress database size:
var dbInstanceSize = config.Get("dbInstanceSize") ?? "db.t3.small";
// The WordPress database name:
var dbName = config.Get("dbName") ?? "wordpressdb";
// The WordPress database user's name:
var dbUsername = config.Get("dbUsername") ?? "admin";
// The WordPress database user's password:
var dbPassword = config.RequireSecret("dbPassword");
// The WordPress EC2 instance's size:
var ec2InstanceSize = config.Get("ec2InstanceSize") ?? "t3.small";

configuration:
 # A path to the EC2 keypair's public key:
 publicKeyPath:
 type: String
 required: true
 # A path to the EC2 keypair's private key:
 privateKeyPath:
 type: String
 required: true
 # The WordPress database size:
 dbInstanceSize:
 type: String
 default: db.t3.small
 # The WordPress database name:
 dbName:
 type: String
 default: wordpressdb
 # The WordPress database user's name:
 dbUsername:
 type: String
 default: admin
 # The WordPress database user's password:
 dbPassword:
 type: String
 secret: true
 required: true
 # The WordPress EC2 instance's size:
 ec2InstanceSize:
 type: String
 default: t3.small

After this, you’ll declare a few variables that are fetched dynamically, so you don’t need to hard-code them, and so your stack can be flexibly deployed into different AWS regions. This includes automatically getting the availability zones in the target AWS region as well as the AMI for Amazon Linux 2 which also varies by region:

// Dynamically fetch AZs so we can spread across them.
const availabilityZones = aws.getAvailabilityZones();
// Dynamically query for the Amazon Linux 2 AMI in this region.
const awsLinuxAmi = aws.ec2.getAmi({
 owners: ["amazon"],
 filters: [{
 name: "name",
 values: ["amzn2-ami-hvm-*-x86_64-ebs"],
 }],
 mostRecent: true,
});

# Dynamically fetch AZs so we can spread across them.
availability_zones = aws.get_availability_zones()
# Dynamically query for the Amazon Linux 2 AMI in this region.
aws_linux_ami = aws.ec2.get_ami(owners=["amazon"],
 filters=[aws.ec2.GetAmiFilterArgs(
 name="name",
 values=["amzn2-ami-hvm-*-x86_64-ebs"],
 )],
 most_recent=True)

// Dynamically fetch AZs so we can spread across them.
availabilityZones, err := aws.GetAvailabilityZones(ctx, nil)
if err != nil {
 return err
}
// Dynamically query for the Amazon Linux 2 AMI in this region.
awsLinuxAmi, err := ec2.LookupAmi(ctx, &ec2.LookupAmiArgs{
 Owners: []string{
 "amazon",
 },
 Filters: []ec2.GetAmiFilter{
 ec2.GetAmiFilter{
 Name: "name",
 Values: []string{
 "amzn2-ami-hvm-*-x86_64-ebs",
 },
 },
 },
 MostRecent: pulumi.BoolRef(true),
}, nil)
if err != nil {
 return err
}

// Dynamically fetch AZs so we can spread across them.
var availabilityZones = Aws.GetAvailabilityZones.Invoke();
// Dynamically query for the Amazon Linux 2 AMI in this region.
var awsLinuxAmi = Aws.Ec2.GetAmi.Invoke(new Aws.Ec2.GetAmiInvokeArgs
{
 Owners = { "amazon" },
 Filters =
 {
 new Aws.Ec2.Inputs.GetAmiFilterInputArgs
 {
 Name = "name",
 Values = { "amzn2-ami-hvm-*-x86_64-ebs" },
 },
 },
 MostRecent = true,
});

variables:
 # Dynamically fetch AZs so we can spread across them.
 availabilityZones:
 Fn::Invoke:
 Function: aws:getAvailabilityZones
 # Dynamically query for the Amazon Linux 2 AMI in this region.
 awsLinuxAmi:
 Fn::Invoke:
 Function: aws:ec2:getAmi
 Arguments:
 owners: ["amazon"]
 filters:
 - name: name
 values: ["amzn2-ami-hvm-*-x86_64-ebs"]
 mostRecent: true

Finally, you’ll read in the contents of the public and private key files so they’re ready to use in the infrastructure declarations that are coming soon. Notice that this code uses Pulumi’s built-in secrets feature to ensure the private key remains encrypted at all times:

// Read in the public key for easy use below.
const publicKey = fs.readFileSync(publicKeyPath).toString();
// Read in the private key for easy use below (and to ensure it's marked a secret!)
const privateKey = pulumi.secret(fs.readFileSync(privateKeyPath).toString());

# Read in the public key for easy use below.
public_key = open(public_key_path).read()
# Read in the private key for easy use below (and to ensure it's marked a secret!)
private_key = pulumi.Output.secret(open(private_key_path).read())

// Read in the public key for easy use below.
publicKeyBytes, err := ioutil.ReadFile(publicKeyPath)
if err != nil {
 return err
}
publicKey := pulumi.String(string(publicKeyBytes))
// Read in the private key for easy use below (and to ensure it's marked a secret!)
privateKeyBytes, err := ioutil.ReadFile(privateKeyPath)
if err != nil {
 return err
}
privateKey := pulumi.ToSecret(string(privateKeyBytes))

// Read in the public key for easy use below.
var publicKey = File.ReadAllText(publicKeyPath);
// Read in the private key for easy use below (and to ensure it's marked a secret!)
var privateKey = Output.CreateSecret(File.ReadAllText(privateKeyPath));

variables:
 ...
 # Read in the public key for easy use below.
 publicKey:
 Fn::ReadFile: ${publicKeyPath}
 # Read in the private key for easy use below (and to ensure it's marked a secret!)
 privateKey:
 Fn::Secret:
 Fn::ReadFile: ${privateKeyPath}

Next up, you will start declaring the AWS infrastructure building blocks needed to run your WordPress server.

Provisioning the AWS Network

First comes the network. This is the network into which you’ll deploy the WordPress database and VM. The VM needs to be accessible over the Internet so that you can access WordPress over HTTP(S). But the database clearly must not be! Only the VM will be able to access it. Accomplishing this first entails provisioning a VPC, public and private subnets, an Internet gateway, and route tables:

// Set up a Virtual Private Cloud to deploy our EC2 instance and RDS database into.
const prodVpc = new aws.ec2.Vpc("prod-vpc", {
 cidrBlock: "10.192.0.0/16",
 enableDnsSupport: true, // gives you an internal domain name.
 enableDnsHostnames: true, // gives you an internal host name.
 enableClassiclink: false,
 instanceTenancy: "default",
});

// Create public subnets for the EC2 instance.
const prodSubnetPublic1 = new aws.ec2.Subnet("prod-subnet-public-1", {
 vpcId: prodVpc.id,
 cidrBlock: "10.192.0.0/24",
 mapPublicIpOnLaunch: true, // public
 availabilityZone: availabilityZones.then(azs => azs.names[0]),
});

// Create private subnets for RDS:
const prodSubnetPrivate1 = new aws.ec2.Subnet("prod-subnet-private-1", {
 vpcId: prodVpc.id,
 cidrBlock: "10.192.20.0/24",
 mapPublicIpOnLaunch: false, // private
 availabilityZone: availabilityZones.then(azs => azs.names[1]),
});
const prodSubnetPrivate2 = new aws.ec2.Subnet("prod-subnet-private-2", {
 vpcId: prodVpc.id,
 cidrBlock: "10.192.21.0/24",
 mapPublicIpOnLaunch: false, // private
 availabilityZone: availabilityZones.then(azs => azs.names[2]),
});

// Create a gateway for internet connectivity:
const prodIgw = new aws.ec2.InternetGateway("prod-igw", {vpcId: prodVpc.id});

// Create and associate a route table:
const prodPublicRt = new aws.ec2.RouteTable("prod-public-rt", {
 vpcId: prodVpc.id,
 routes: [{
 // associated subnets can reach anywhere.
 cidrBlock: "0.0.0.0/0",
 // use this IGW to reach the internet.
 gatewayId: prodIgw.id,
 }],
});
const prodRtaPublicSubnet1 = new aws.ec2.RouteTableAssociation("prod-rta-public-subnet-1", {
 subnetId: prodSubnetPublic1.id,
 routeTableId: prodPublicRt.id,
});

# Set up a Virtual Private Cloud to deploy our EC2 instance and RDS database into.
prod_vpc = aws.ec2.Vpc("prod-vpc",
 cidr_block="10.192.0.0/16",
 enable_dns_support=True, # gives you an internal domain name
 enable_dns_hostnames=True, # gives yoiu an internal host name
 enable_classiclink=False,
 instance_tenancy="default")

# Create public subnets for the EC2 instance.
prod_subnet_public1 = aws.ec2.Subnet("prod-subnet-public-1",
 vpc_id=prod_vpc.id,
 cidr_block="10.192.0.0/24",
 map_public_ip_on_launch=True,
 availability_zone=availability_zones.names[0])

# Create private subnets for RDS:
prod_subnet_private1 = aws.ec2.Subnet("prod-subnet-private-1",
 vpc_id=prod_vpc.id,
 cidr_block="10.192.20.0/24",
 map_public_ip_on_launch=False,
 availability_zone=availability_zones.names[1])
prod_subnet_private2 = aws.ec2.Subnet("prod-subnet-private-2",
 vpc_id=prod_vpc.id,
 cidr_block="10.192.21.0/24",
 map_public_ip_on_launch=False,
 availability_zone=availability_zones.names[2])

# Create a gateway for internet connectivity:
prod_igw = aws.ec2.InternetGateway("prod-igw", vpc_id=prod_vpc.id)

# Create a route table:
prod_public_rt = aws.ec2.RouteTable("prod-public-rt",
 vpc_id=prod_vpc.id,
 routes=[aws.ec2.RouteTableRouteArgs(
 # associated subnets can reach anywhere:
 cidr_block="0.0.0.0/0",
 # use this IGW to reach the internet:
 gateway_id=prod_igw.id,
 )])
prod_rta_public_subnet1 = aws.ec2.RouteTableAssociation("prod-rta-public-subnet-1",
 subnet_id=prod_subnet_public1.id,
 route_table_id=prod_public_rt.id)

// Set up a Virtual Private Cloud to deploy our EC2 instance and RDS database into.
prodVpc, err := ec2.NewVpc(ctx, "prod-vpc", &ec2.VpcArgs{
 CidrBlock: pulumi.String("10.192.0.0/16"),
 EnableDnsSupport: pulumi.Bool(true), // gives you an internal domain name.
 EnableDnsHostnames: pulumi.Bool(true), // gives you an internal host name.
 EnableClassiclink: pulumi.Bool(false),
 InstanceTenancy: pulumi.String("default"),
})
if err != nil {
 return err
}
// Create public subnets for the EC2 instance.
prodSubnetPublic1, err := ec2.NewSubnet(ctx, "prod-subnet-public-1", &ec2.SubnetArgs{
 VpcId: prodVpc.ID(),
 CidrBlock: pulumi.String("10.192.0.0/24"),
 MapPublicIpOnLaunch: pulumi.Bool(true), // public
 AvailabilityZone: pulumi.String(availabilityZones.Names[0]),
})
if err != nil {
 return err
}
// Create private subnets for RDS:
prodSubnetPrivate1, err := ec2.NewSubnet(ctx, "prod-subnet-private-1", &ec2.SubnetArgs{
 VpcId: prodVpc.ID(),
 CidrBlock: pulumi.String("10.192.20.0/24"),
 MapPublicIpOnLaunch: pulumi.Bool(false), // private
 AvailabilityZone: pulumi.String(availabilityZones.Names[1]),
})
if err != nil {
 return err
}
prodSubnetPrivate2, err := ec2.NewSubnet(ctx, "prod-subnet-private-2", &ec2.SubnetArgs{
 VpcId: prodVpc.ID(),
 CidrBlock: pulumi.String("10.192.21.0/24"),
 MapPublicIpOnLaunch: pulumi.Bool(false), // private
 AvailabilityZone: pulumi.String(availabilityZones.Names[2]),
})
if err != nil {
 return err
}
// Create a gateway for internet connectivity:
prodIgw, err := ec2.NewInternetGateway(ctx, "prod-igw", &ec2.InternetGatewayArgs{
 VpcId: prodVpc.ID(),
})
if err != nil {
 return err
}
// Create a route table:
prodPublicRt, err := ec2.NewRouteTable(ctx, "prod-public-rt", &ec2.RouteTableArgs{
 VpcId: prodVpc.ID(),
 Routes: ec2.RouteTableRouteArray{
 &ec2.RouteTableRouteArgs{
 // associated subnets can reach anywhere.
 CidrBlock: pulumi.String("0.0.0.0/0"),
 // use this IGW to reach the internet.
 GatewayId: prodIgw.ID(),
 },
 },
})
if err != nil {
 return err
}
_, err = ec2.NewRouteTableAssociation(ctx, "prod-rta-public-subnet-1", &ec2.RouteTableAssociationArgs{
 SubnetId: prodSubnetPublic1.ID(),
 RouteTableId: prodPublicRt.ID(),
})
if err != nil {
 return err
}

// Set up a Virtual Private Cloud to deploy our EC2 instance and RDS database into.
var prodVpc = new Aws.Ec2.Vpc("prod-vpc", new Aws.Ec2.VpcArgs
{
 CidrBlock = "10.192.0.0/16",
 EnableDnsSupport = true, // gives you an internal domain name.
 EnableDnsHostnames = true, // gives you an internal host name.
 EnableClassiclink = false,
 InstanceTenancy = "default",
});

// Create public subnets for the EC2 instance.
var prodSubnetPublic1 = new Aws.Ec2.Subnet("prod-subnet-public-1", new Aws.Ec2.SubnetArgs
{
 VpcId = prodVpc.Id,
 CidrBlock = "10.192.0.0/24",
 MapPublicIpOnLaunch = true, // public
 AvailabilityZone = availabilityZones.Apply(az => az.Names[0]),
});

// Create private subnets for RDS:
var prodSubnetPrivate1 = new Aws.Ec2.Subnet("prod-subnet-private-1", new Aws.Ec2.SubnetArgs
{
 VpcId = prodVpc.Id,
 CidrBlock = "10.192.20.0/24",
 MapPublicIpOnLaunch = false, // private
 AvailabilityZone = availabilityZones.Apply(az => az.Names[1]),
});
var prodSubnetPrivate2 = new Aws.Ec2.Subnet("prod-subnet-private-2", new Aws.Ec2.SubnetArgs
{
 VpcId = prodVpc.Id,
 CidrBlock = "10.192.21.0/24",
 MapPublicIpOnLaunch = false, // private
 AvailabilityZone = availabilityZones.Apply(az => az.Names[2]),
});

// Create a gateway for internet connectivity:
var prodIgw = new Aws.Ec2.InternetGateway("prod-igw", new Aws.Ec2.InternetGatewayArgs
{
 VpcId = prodVpc.Id,
});

// Create and associate a route table:
var prodPublicCrt = new Aws.Ec2.RouteTable("prod-public-crt", new Aws.Ec2.RouteTableArgs
{
 VpcId = prodVpc.Id,
 Routes =
 {
 new Aws.Ec2.Inputs.RouteTableRouteArgs
 {
 // associated subnets can reach anywhere.
 CidrBlock = "0.0.0.0/0",
 // use this IGW to reach the internet.
 GatewayId = prodIgw.Id,
 },
 },
});
var prodCrtaPublicSubnet1 = new Aws.Ec2.RouteTableAssociation("prod-crta-public-subnet-1", new Aws.Ec2.RouteTableAssociationArgs
{
 SubnetId = prodSubnetPublic1.Id,
 RouteTableId = prodPublicCrt.Id,
});

resources:
 # Set up a Virtual Private Cloud to deploy our EC2 instance and RDS database into.
 prod-vpc:
 type: aws:ec2:Vpc
 properties:
 cidrBlock: 10.192.0.0/16
 enableDnsSupport: true # gives you an internal domain name
 enableDnsHostnames: true # gives you an internal host name
 enableClassiclink: false
 instanceTenancy: default
 # Create public subnets for the EC2 instance.
 prod-subnet-public-1:
 type: aws:ec2:Subnet
 properties:
 vpcId: ${prod-vpc.id}
 cidrBlock: 10.192.0.0/24
 mapPublicIpOnLaunch: true # public subnet
 availabilityZone: ${availabilityZones.names[0]}
 # Create private subnets for RDS:
 prod-subnet-private-1:
 type: aws:ec2:Subnet
 properties:
 vpcId: ${prod-vpc.id}
 cidrBlock: 10.192.20.0/24
 mapPublicIpOnLaunch: false # private subnet
 availabilityZone: ${availabilityZones.names[1]}
 prod-subnet-private-2:
 type: aws:ec2:Subnet
 properties:
 vpcId: ${prod-vpc.id}
 cidrBlock: 10.192.21.0/24
 mapPublicIpOnLaunch: false # private subnet
 availabilityZone: ${availabilityZones.names[2]}
 # Create a gateway for internet connectivity:
 prod-igw:
 type: aws:ec2:InternetGateway
 properties:
 vpcId: ${prod-vpc.id}
 # Create a route table:
 prod-public-rt:
 type: aws:ec2:RouteTable
 properties:
 vpcId: ${prod-vpc.id}
 routes:
 # associated subnets can reach anywhere:
 - cidrBlock: 0.0.0.0/0
 # we use this IGW to reach the internet:
 gatewayId: ${prod-igw.id}
 prod-rta-public-subnet-1:
 type: aws:ec2:RouteTableAssociation
 properties:
 subnetId: ${prod-subnet-public-1.id}
 routeTableId: ${prod-public-rt.id}

Note: All of the AWS building blocks are declared one at a time here to make it entirely transparent what’s going on. Once you’ve set up a VPC a handful of times, however, this can get tedious. The Pulumi Crosswalk for AWS package offers standard VPC configurations out-of-the-box with fewer lines of code, and built-in best practices. Check it out!

Next, to accomplish the stated connectivity goals — EC2 instance available over HTTP(S) and SSH on the Internet, and the RDS database available only to MySQL traffic originating from the EC2 instance itself – you’ll now create two security groups, one each for the public and private subnets respectively:

// Security group for EC2:
const ec2AllowRule = new aws.ec2.SecurityGroup("ec2-allow-rule", {
 vpcId: prodVpc.id,
 ingress: [
 {
 description: "HTTPS",
 fromPort: 443,
 toPort: 443,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 {
 description: "HTTP",
 fromPort: 80,
 toPort: 80,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 {
 description: "SSH",
 fromPort: 22,
 toPort: 22,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
 egress: [{
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 }],
 tags: {
 Name: "allow ssh,http,https",
 },
});

// Security group for RDS:
const rdsAllowRule = new aws.ec2.SecurityGroup("rds-allow-rule", {
 vpcId: prodVpc.id,
 ingress: [{
 description: "MySQL",
 fromPort: 3306,
 toPort: 3306,
 protocol: "tcp",
 securityGroups: [ec2AllowRule.id],
 }],
 // Allow all outbound traffic.
 egress: [{
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 }],
 tags: {
 Name: "allow ec2",
 },
});

# Security group for EC2:
ec2_allow_rule = aws.ec2.SecurityGroup("ec2-allow-rule",
 vpc_id=prod_vpc.id,
 ingress=[
 aws.ec2.SecurityGroupIngressArgs(
 description="HTTPS",
 from_port=443,
 to_port=443,
 protocol="tcp",
 cidr_blocks=["0.0.0.0/0"],
 ),
 aws.ec2.SecurityGroupIngressArgs(
 description="HTTP",
 from_port=80,
 to_port=80,
 protocol="tcp",
 cidr_blocks=["0.0.0.0/0"],
 ),
 aws.ec2.SecurityGroupIngressArgs(
 description="SSH",
 from_port=22,
 to_port=22,
 protocol="tcp",
 cidr_blocks=["0.0.0.0/0"],
 ),
 ],
 egress=[aws.ec2.SecurityGroupEgressArgs(
 from_port=0,
 to_port=0,
 protocol="-1",
 cidr_blocks=["0.0.0.0/0"],
 )],
 tags={
 "Name": "allow ssh,http,https",
 })

# Security group for RDS:
rds_allow_rule = aws.ec2.SecurityGroup("rds-allow-rule",
 vpc_id=prod_vpc.id,
 ingress=[aws.ec2.SecurityGroupIngressArgs(
 description="MySQL",
 from_port=3306,
 to_port=3306,
 protocol="tcp",
 security_groups=[ec2_allow_rule.id],
 )],
 # allow all outbound traffic.
 egress=[aws.ec2.SecurityGroupEgressArgs(
 from_port=0,
 to_port=0,
 protocol="-1",
 cidr_blocks=["0.0.0.0/0"],
 )],
 tags={
 "Name": "allow ec2",
 })

// Security group for EC2:
ec2AllowRule, err := ec2.NewSecurityGroup(ctx, "ec2-allow-rule", &ec2.SecurityGroupArgs{
 VpcId: prodVpc.ID(),
 Ingress: ec2.SecurityGroupIngressArray{
 &ec2.SecurityGroupIngressArgs{
 Description: pulumi.String("HTTPS"),
 FromPort: pulumi.Int(443),
 ToPort: pulumi.Int(443),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{
 pulumi.String("0.0.0.0/0"),
 },
 },
 &ec2.SecurityGroupIngressArgs{
 Description: pulumi.String("HTTP"),
 FromPort: pulumi.Int(80),
 ToPort: pulumi.Int(80),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{
 pulumi.String("0.0.0.0/0"),
 },
 },
 &ec2.SecurityGroupIngressArgs{
 Description: pulumi.String("SSH"),
 FromPort: pulumi.Int(22),
 ToPort: pulumi.Int(22),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{
 pulumi.String("0.0.0.0/0"),
 },
 },
 },
 Egress: ec2.SecurityGroupEgressArray{
 &ec2.SecurityGroupEgressArgs{
 FromPort: pulumi.Int(0),
 ToPort: pulumi.Int(0),
 Protocol: pulumi.String("-1"),
 CidrBlocks: pulumi.StringArray{
 pulumi.String("0.0.0.0/0"),
 },
 },
 },
 Tags: pulumi.StringMap{
 "Name": pulumi.String("allow ssh,http,https"),
 },
})
if err != nil {
 return err
}
// Security group for RDS:
rdsAllowRule, err := ec2.NewSecurityGroup(ctx, "rds-allow-rule", &ec2.SecurityGroupArgs{
 VpcId: prodVpc.ID(),
 Ingress: ec2.SecurityGroupIngressArray{
 &ec2.SecurityGroupIngressArgs{
 Description: pulumi.String("MySQL"),
 FromPort: pulumi.Int(3306),
 ToPort: pulumi.Int(3306),
 Protocol: pulumi.String("tcp"),
 SecurityGroups: pulumi.StringArray{
 ec2AllowRule.ID(),
 },
 },
 },
 // Allow all outbound traffic.
 Egress: ec2.SecurityGroupEgressArray{
 &ec2.SecurityGroupEgressArgs{
 FromPort: pulumi.Int(0),
 ToPort: pulumi.Int(0),
 Protocol: pulumi.String("-1"),
 CidrBlocks: pulumi.StringArray{
 pulumi.String("0.0.0.0/0"),
 },
 },
 },
 Tags: pulumi.StringMap{
 "Name": pulumi.String("allow ec2"),
 },
})
if err != nil {
 return err
}

// Security group for EC2:
var ec2AllowRule = new Aws.Ec2.SecurityGroup("ec2-allow-rule", new Aws.Ec2.SecurityGroupArgs
{
 VpcId = prodVpc.Id,
 Ingress =
 {
 new Aws.Ec2.Inputs.SecurityGroupIngressArgs
 {
 Description = "HTTPS",
 FromPort = 443,
 ToPort = 443,
 Protocol = "tcp",
 CidrBlocks =
 {
 "0.0.0.0/0",
 },
 },
 new Aws.Ec2.Inputs.SecurityGroupIngressArgs
 {
 Description = "HTTP",
 FromPort = 80,
 ToPort = 80,
 Protocol = "tcp",
 CidrBlocks =
 {
 "0.0.0.0/0",
 },
 },
 new Aws.Ec2.Inputs.SecurityGroupIngressArgs
 {
 Description = "SSH",
 FromPort = 22,
 ToPort = 22,
 Protocol = "tcp",
 CidrBlocks =
 {
 "0.0.0.0/0",
 },
 },
 },
 Egress =
 {
 new Aws.Ec2.Inputs.SecurityGroupEgressArgs
 {
 FromPort = 0,
 ToPort = 0,
 Protocol = "-1",
 CidrBlocks =
 {
 "0.0.0.0/0",
 },
 },
 },
 Tags =
 {
 { "Name", "allow ssh,http,https" },
 },
});

// Security group for RDS:
var rdsAllowRule = new Aws.Ec2.SecurityGroup("rds-allow-rule", new Aws.Ec2.SecurityGroupArgs
{
 VpcId = prodVpc.Id,
 Ingress =
 {
 new Aws.Ec2.Inputs.SecurityGroupIngressArgs
 {
 Description = "MySQL",
 FromPort = 3306,
 ToPort = 3306,
 Protocol = "tcp",
 SecurityGroups =
 {
 ec2AllowRule.Id,
 },
 },
 },
 // Allow all outbound traffic.
 Egress =
 {
 new Aws.Ec2.Inputs.SecurityGroupEgressArgs
 {
 FromPort = 0,
 ToPort = 0,
 Protocol = "-1",
 CidrBlocks =
 {
 "0.0.0.0/0",
 },
 },
 },
 Tags =
 {
 { "Name", "allow ec2" },
 },
});

 # Security group for EC2:
 ec2-allow-rule:
 type: aws:ec2:SecurityGroup
 properties:
 vpcId: ${prod-vpc.id}
 ingress:
 - description: "HTTPS"
 fromPort: 443
 toPort: 443
 protocol: "tcp"
 cidrBlocks: [0.0.0.0/0]
 - description: "HTTP"
 fromPort: 80
 toPort: 80
 protocol: "tcp"
 cidrBlocks: [0.0.0.0/0]
 - description: "SSH"
 fromPort: 22
 toPort: 22
 protocol: "tcp"
 cidrBlocks: [0.0.0.0/0]
 egress:
 - fromPort: 0
 toPort: 0
 protocol: "-1"
 cidrBlocks: [0.0.0.0/0]
 tags:
 Name: "allow ssh,http,https"
 # Security group for RDS:
 rds-allow-rule:
 type: aws:ec2:SecurityGroup
 properties:
 vpcId: ${prod-vpc.id}
 # Only allow MySQL ingress from the EC2 VM.
 ingress:
 - description: "MySQL"
 fromPort: 3306
 toPort: 3306
 protocol: "tcp"
 securityGroups: ["${ec2-allow-rule.id}"]
 # Allow all outbound traffic.
 egress:
 - fromPort: 0
 toPort: 0
 protocol: "-1"
 cidrBlocks: [0.0.0.0/0]
 tags:
 Name: "allow ec2"

Now that you have a network, you will deploy the WordPress RDS database and the EC2 instance into it.

Provisioning the AWS EC2 Instance and RDS Database

To start, create a RDS database running MySQL 5.7 so that WordPress can use it. The database is given a specific database name, username, and password, so that WordPress can be subsequently configured to access it using this information. You’ll also create a subnet group to place RDS into the private subnets allocated for it earlier:

// Place the RDS instance into private subnets:
const rdsSubnetGrp = new aws.rds.SubnetGroup("rds-subnet-grp", {subnetIds: [
 prodSubnetPrivate1.id,
 prodSubnetPrivate2.id,
]});

// Create the RDS instance:
const wordpressdb = new aws.rds.Instance("wordpressdb", {
 allocatedStorage: 10,
 engine: "mysql",
 engineVersion: "5.7",
 instanceClass: dbInstanceSize,
 dbSubnetGroupName: rdsSubnetGrp.id,
 vpcSecurityGroupIds: [rdsAllowRule.id],
 dbName: dbName,
 username: dbUsername,
 password: dbPassword,
 skipFinalSnapshot: true,
});

# Place the RDS instance into private subnets:
rds_subnet_grp = aws.rds.SubnetGroup("rds-subnet-grp", subnet_ids=[
 prod_subnet_private1.id,
 prod_subnet_private2.id,
])

# Create the RDS instance:
wordpressdb = aws.rds.Instance("wordpressdb",
 allocated_storage=10,
 engine="mysql",
 engine_version="5.7",
 instance_class=db_instance_size,
 db_subnet_group_name=rds_subnet_grp.id,
 vpc_security_group_ids=[rds_allow_rule.id],
 db_name=db_name,
 username=db_username,
 password=db_password,
 skip_final_snapshot=True)

// Place the RDS instance into private subnets:
rdsSubnetGrp, err := rds.NewSubnetGroup(ctx, "rds-subnet-grp", &rds.SubnetGroupArgs{
 SubnetIds: pulumi.StringArray{
 prodSubnetPrivate1.ID(),
 prodSubnetPrivate2.ID(),
 },
})
if err != nil {
 return err
}
// Create the RDS instance:
wordpressdb, err := rds.NewInstance(ctx, "wordpressdb", &rds.InstanceArgs{
 AllocatedStorage: pulumi.Int(10),
 Engine: pulumi.String("mysql"),
 EngineVersion: pulumi.String("5.7"),
 InstanceClass: pulumi.String(dbInstanceSize),
 DbSubnetGroupName: rdsSubnetGrp.ID(),
 VpcSecurityGroupIds: pulumi.StringArray{
 rdsAllowRule.ID(),
 },
 DbName: pulumi.String(dbName),
 Username: pulumi.String(dbUsername),
 Password: dbPassword,
 SkipFinalSnapshot: pulumi.Bool(true),
})
if err != nil {
 return err
}

// Place the RDS instance into private subnets:
var rdsSubnetGrp = new Aws.Rds.SubnetGroup("rds-subnet-grp", new Aws.Rds.SubnetGroupArgs
{
 SubnetIds =
 {
 prodSubnetPrivate1.Id,
 prodSubnetPrivate2.Id,
 },
});

// Create the RDS instance:
var wordpressdb = new Aws.Rds.Instance("wordpressdb", new Aws.Rds.InstanceArgs
{
 AllocatedStorage = 10,
 Engine = "mysql",
 EngineVersion = "5.7",
 InstanceClass = dbInstanceSize,
 DbSubnetGroupName = rdsSubnetGrp.Id,
 VpcSecurityGroupIds =
 {
 rdsAllowRule.Id,
 },
 DbName = dbName,
 Username = dbUsername,
 Password = dbPassword,
 SkipFinalSnapshot = true,
});

 # Place RDS into the private subnets:
 rds-subnet-grp:
 type: aws:rds:SubnetGroup
 properties:
 subnetIds:
 - ${prod-subnet-private-1.id}
 - ${prod-subnet-private-2.id}
 # Create the RDS instance:
 wordpressdb:
 type: aws:rds:Instance
 properties:
 allocatedStorage: 10
 engine: "mysql"
 engineVersion: "5.7"
 instanceClass: ${dbInstanceSize}
 dbSubnetGroupName: ${rds-subnet-grp.id}
 vpcSecurityGroupIds: ["${rds-allow-rule.id}"]
 dbName: ${dbName}
 username: ${dbUsername}
 password: ${dbPassword}
 skipFinalSnapshot: true

Now that you have a database, you’re ready to provision the EC2 instance and supporting infrastructure. This step comes after the RDS database, since WordPress running on the instance needs to access the database. Pulumi tracks the dependencies between them so it performs operations in the correct order.

Start by creating an EC2 key pair so you can SSH into the machine. This is also required to execute remote commands during the post-provisioning configuration:

// Create a keypair to access the EC2 instance:
const wordpressKeypair = new aws.ec2.KeyPair("wordpress-keypair", {
 publicKey: publicKey,
});

# Create a keypair to access the EC2 instance:
wordpress_keypair = aws.ec2.KeyPair("wordpress-keypair", public_key=public_key)

// Create a keypair to access the EC2 instance:
wordpressKeypair, err := ec2.NewKeyPair(ctx, "wordpress-keypair", &ec2.KeyPairArgs{
 PublicKey: pulumi.String(publicKey),
})
if err != nil {
 return err
}

// Create a keypair to access the EC2 instance:
var wordpressKeypair = new Aws.Ec2.KeyPair("wordpress-keypair", new Aws.Ec2.KeyPairArgs
{
 PublicKey = publicKey,
});

 # Create a keypair to access the EC2 instance:
 wordpress-keypair:
 type: aws:ec2:KeyPair
 properties:
 publicKey: ${publicKey}

Note: Enabling operators to manually SSH into machines is a common practice. Most of us are generally trying to move away from it these days, however. SSH’ing into a machine and manually running commands just leads to headaches: configuration drift, unrepeatable automation, and deployment outages, to name a few. All of these are antithetical to the modern way of doing cloud engineering – but no doubt, it’s unfortunately still relatively commonplace.

Now create the EC2 instance and give it an elastic IP (EIP) address so that it’s stable even if you need to upgrade or even entirely replace the server:

// Create an EC2 instance to run Wordpress (after RDS is ready).
const wordpressInstance = new aws.ec2.Instance("wordpress-instance", {
 ami: awsLinuxAmi.then(awsLinuxAmi => awsLinuxAmi.id),
 instanceType: ec2InstanceSize,
 subnetId: prodSubnetPublic1.id,
 vpcSecurityGroupIds: [ec2AllowRule.id],
 keyName: wordpressKeypair.id,
 tags: {
 Name: "Wordpress.web",
 },
}, {
 // Only create after RDS is provisioned.
 dependsOn: [wordpressdb],
});

// Give our EC2 instance an elastic IP address.
const wordpressEip = new aws.ec2.Eip("wordpress-eip", {
 instance: wordpressInstance.id,
});

# Create an EC2 instance to run Wordpress (after RDS is ready).
wordpress_instance = aws.ec2.Instance("wordpress-instance",
 ami=aws_linux_ami.id,
 instance_type=ec2_instance_size,
 subnet_id=prod_subnet_public1.id,
 vpc_security_group_ids=[ec2_allow_rule.id],
 key_name=wordpress_keypair.id,
 tags={
 "Name": "Wordpress.web",
 },
 # Only create after RDS is provisioned.
 opts=pulumi.ResourceOptions(depends_on=[wordpressdb]))

# Give our EC2 instance an elastic IP address.
wordpress_eip = aws.ec2.Eip("wordpress-eip", instance=wordpress_instance.id)

// Create an EC2 instance to run Wordpress (after RDS is ready).
wordpressInstance, err := ec2.NewInstance(ctx, "wordpress-instance", &ec2.InstanceArgs{
 Ami: pulumi.String(awsLinuxAmi.Id),
 InstanceType: pulumi.String(ec2InstanceSize),
 SubnetId: prodSubnetPublic1.ID(),
 VpcSecurityGroupIds: pulumi.StringArray{
 ec2AllowRule.ID(),
 },
 KeyName: wordpressKeypair.ID(),
 Tags: pulumi.StringMap{
 "Name": pulumi.String("Wordpress.web"),
 },
}, pulumi.DependsOn([]pulumi.Resource{
 // Only create after RDS is provisioned.
 wordpressdb,
}))
if err != nil {
 return err
}

// Give our EC2 instance an elastic IP address.
wordpressEip, err := ec2.NewEip(ctx, "wordpress-eip", &ec2.EipArgs{
 Instance: wordpressInstance.ID(),
})
if err != nil {
 return err
}

// Create an EC2 instance to run Wordpress (after RDS is ready).
var wordpressInstance = new Aws.Ec2.Instance("wordpress-instance", new Aws.Ec2.InstanceArgs
{
 Ami = awsLinuxAmi.Apply(awsLinuxAmi => awsLinuxAmi.Id),
 InstanceType = ec2InstanceSize,
 SubnetId = prodSubnetPublic1.Id,
 VpcSecurityGroupIds =
 {
 ec2AllowRule.Id,
 },
 KeyName = wordpressKeypair.Id,
 Tags =
 {
 { "Name", "Wordpress.web" },
 },
}, new CustomResourceOptions
{
 // Only create after RDS is provisioned.
 DependsOn = { wordpressdb },
});

// Give our EC2 instance an elastic IP address.
var wordpressEip = new Aws.Ec2.Eip("wordpress-eip", new Aws.Ec2.EipArgs
{
 Instance = wordpressInstance.Id,
});

 # Create an EC2 instance to run Wordpress (after RDS is ready).
 wordpress-instance:
 type: aws:ec2:Instance
 properties:
 ami: ${awsLinuxAmi.id}
 instanceType: ${ec2InstanceSize}
 subnetId: ${prod-subnet-public-1.id}
 vpcSecurityGroupIds: ["${ec2-allow-rule.id}"]
 keyName: ${wordpress-keypair.id}
 tags:
 Name: "Wordpress.web"
 options:
 dependsOn:
 # Only create after RDS is provisioned.
 - ${wordpressdb}
 # Give our EC2 instance an elastic IP address.
 wordpress-eip:
 type: aws:ec2:Eip
 properties:
 instance: ${wordpress-instance.id}

Configuring the AWS EC2 Instance Using the Pulumi Command Package

Here’s where things deviate from the normal Pulumi way of doing things: we’ll now configure our server.

Pulumi’s Command package lets you run local or remote commands sequenced amongst other ordinary infrastructure as code actions. For instance, you can copy a local file to a remote server using SCP, run a local command, or run a remote command over SSH, before or after creating or destroying other infrastructure. This is precisely the building block that will allow Pulumi to orchestrate configuration management tasks.

The first thing to do is render the Ansible playbook. Why “render”? The playbook.yml needs information only known at deployment time within your Pulumi program, including the server’s address, the database URL, name, username, password, and so on, all of which are configurable and only known after spinning up the server and database.

The playbook.yml contains everything necessary to get Wordpress up and running. Notice the templated variables like $DB_RDS that need to be substituted with real values:

# An Ansible Playbook that installs Wordpress on a fresh Amazon Linux 2 Server.
# The only prerequisite is that Python 2.7 and Ansible are available on the server.
- name: Install Wordpress on new Amazon Linux 2 Server.
 hosts: all
 become: yes

 vars:
 # Explicitly use Python2.7, since this playbook requires it.
 ansible_python_interpreter: /usr/bin/python

 tasks:

 - name: Setting up variables
 set_fact:
 php_modules:
...
 # RDS MySQL variables populated post-provisioning:
 mysql_rds: $DB_RDS
 mysql_db: $DB_NAME
 mysql_user: $DB_USERNAME
 mysql_password: $DB_PASSWORD

 - name: Yum update
...

Note: The playbook file has been truncated for brevity. The full playbook.yml is available here.

To render the playbook, you’ll just use the command:local:Command resource to execute the standard envsubst UNIX command:

// Render the Ansible playbook using RDS info.
const renderPlaybookCmd = new command.local.Command("renderPlaybookCmd", {
 create: "cat playbook.yml | envsubst > playbook_rendered.yml",
 environment: {
 DB_RDS: wordpressdb.endpoint,
 DB_NAME: dbName,
 DB_USERNAME: dbUsername,
 DB_PASSWORD: dbPassword,
 },
});

# Render the Ansible playbook using RDS info.
render_playbook_cmd = command.local.Command("renderPlaybookCmd",
 create="cat playbook.yml | envsubst > playbook_rendered.yml",
 environment={
 "DB_RDS": wordpressdb.endpoint,
 "DB_NAME": db_name,
 "DB_USERNAME": db_username,
 "DB_PASSWORD": db_password,
 })

// Render the Ansible playbook using RDS info.
renderPlaybookCmd, err := local.NewCommand(ctx, "renderPlaybookCmd", &local.CommandArgs{
 Create: pulumi.String("cat playbook.yml | envsubst > playbook_rendered.yml"),
 Environment: pulumi.StringMap{
 "DB_RDS": wordpressdb.Endpoint,
 "DB_NAME": pulumi.String(dbName),
 "DB_USERNAME": pulumi.String(dbUsername),
 "DB_PASSWORD": dbPassword,
 },
})
if err != nil {
 return err
}

// Render the Ansible playbook using RDS info.
var renderPlaybookCmd = new Command.Local.Command("renderPlaybookCmd", new Command.Local.CommandArgs
{
 Create = "cat playbook.yml | envsubst > playbook_rendered.yml",
 Environment =
 {
 { "DB_RDS", wordpressdb.Endpoint },
 { "DB_NAME", dbName },
 { "DB_USERNAME", dbUsername },
 { "DB_PASSWORD", dbPassword },
 },
});

 # Render the Ansible playbook using RDS info.
 renderPlaybookCmd:
 type: command:local:Command
 properties:
 create: "cat playbook.yml | envsubst > playbook_rendered.yml"
 environment:
 DB_RDS: ${wordpressdb.endpoint}
 DB_NAME: ${dbName}
 DB_USERNAME: ${dbUsername}
 DB_PASSWORD: ${dbPassword}

This command runs locally on the machine running Pulumi (the control node in Ansible parlance), which means we’ll end up with a file, playbook_rendered.yml, very similar to playbook.yml above but with variables substituted, after executing it:

# An Ansible Playbook that installs Wordpress on a fresh Amazon Linux 2 Server.
# The only prerequisite is that Python 2.7 and Ansible are available on the server.
- name: Install Wordpress on new Amazon Linux 2 Server.
 hosts: all
 become: yes

 vars:
 # Explicitly use Python2.7, since this playbook requires it.
 ansible_python_interpreter: /usr/bin/python

 tasks:

 - name: Setting up variables
 set_fact:
 php_modules:
...

 # RDS MySQL variables populated post-provisioning:
 mysql_rds: wordpressdb72d0d1x.fep6qenxwgtk.us-west-2.rds.amazonaws.com:3306
 mysql_db: wordpressdb
 mysql_user: admin
 mysql_password: [secret]

 - name: Yum update
...

Now things get more interesting: you will run a remote command over SSH to run yum commands that update packages and ensure Python is available, since Ansible will need it. This is done with the command:remote:Command and it fully understands how and when to connect remotely to the newly provisioned EC2 instance:

// Run a script to update packages on the remote machine.
const updatePythonCmd = new command.remote.Command("updatePythonCmd", {
 connection: {
 host: wordpressEip.publicIp,
 port: 22,
 user: "ec2-user",
 privateKey: privateKey,
 },
 create: `(sudo yum update -y || true);
(sudo yum install python35 -y);
(sudo yum install amazon-linux-extras -y)
`,
});

# Run a script to update packages on the remote machine.
update_python_cmd = command.remote.Command("updatePythonCmd",
 connection=command.remote.ConnectionArgs(
 host=wordpress_eip.public_ip,
 port=22,
 user="ec2-user",
 private_key=private_key,
 ),
 create="""(sudo yum update -y || true);
(sudo yum install python35 -y);
(sudo yum install amazon-linux-extras -y)
""")

// Run a script to update packages on the remote machine.
updatePythonCmd, err := remote.NewCommand(ctx, "updatePythonCmd", &remote.CommandArgs{
 Connection: &remote.ConnectionArgs{
 Host: wordpressEip.PublicIp,
 Port: pulumi.Float64(22),
 User: pulumi.String("ec2-user"),
 PrivateKey: privateKey,
 },
 Create: pulumi.String("(sudo yum update -y || true);" +
 "(sudo yum install python35 -y);" +
 "(sudo yum install amazon-linux-extras -y)\n"),
})
if err != nil {
 return err
}

// Run a script to update packages on the remote machine.
var updatePythonCmd = new Command.Remote.Command("updatePythonCmd", new Command.Remote.CommandArgs
{
 Connection = new Command.Remote.Inputs.ConnectionArgs
 {
 Host = wordpressEip.PublicIp,
 Port = 22,
 User = "ec2-user",
 PrivateKey = privateKey,
 },
 Create = "(sudo yum update -y || true);"+
 "(sudo yum install python35 -y);"+
 "(sudo yum install amazon-linux-extras -y)"
});

 # Run a script to update packages on the remote machine.
 updatePythonCmd:
 type: command:remote:Command
 properties:
 connection:
 host: ${wordpress-eip.publicIp}
 port: 22
 user: ec2-user
 privateKey: ${privateKey}
 create: >
 (sudo yum update -y || true);
 (sudo yum install python35 -y);
 (sudo yum install amazon-linux-extras -y)

This looks similar to the local command execution, except that it uses the remote:Command type instead of local:Command, and also needs the EC2 key pair’s private key to SSH into the server. Pulumi’s dependency system ensures this action is sequenced correctly after the server is running and information required to perform the command is available.

Configuring WordPress on EC2 Using Ansible

Alas, we now reach the pièce de résistance! In this section, you will configure WordPress on the new EC2 instance using Ansible.

A local command will run the rendered Ansible playbook using the ansible-playbook command, effectively turning the machine running Pulumi into an Ansible control node. Ansible will chug away on the playbook, executing commands as specified within it, thereby installing WordPress on your new server:

// Finally, play the Ansible playbook to finish installing.
const playAnsiblePlaybookCmd = new command.local.Command("playAnsiblePlaybookCmd", {
 create: pulumi.interpolate`ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook \
-u ec2-user -i '${wordpressEip.publicIp},' \
--private-key ${privateKeyPath} \
playbook_rendered.yml`
}, {
 dependsOn: [
 renderPlaybookCmd,
 updatePythonCmd,
 ],
});

# Finally, play the Ansible playbook to finish installing.
play_ansible_playbook_cmd = command.local.Command("playAnsiblePlaybookCmd",
 create=wordpress_eip.public_ip.apply(lambda public_ip: f"""\
ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook \
-u ec2-user \
-i '{public_ip},' \
--private-key {private_key_path} \
playbook_rendered.yml"""),
opts=pulumi.ResourceOptions(depends_on=[
 render_playbook_cmd,
 update_python_cmd,
 ]))

// Finally, play the Ansible playbook to finish installing.
_, err = local.NewCommand(ctx, "playAnsiblePlaybookCmd", &local.CommandArgs{
 Create: wordpressEip.PublicIp.ApplyT(func(publicIp string) (string, error) {
 return fmt.Sprintf("ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook "+
 "-u ec2-user "+
 "-i '%v,' "+
 "--private-key %v "+
 "playbook_rendered.yml", publicIp, privateKeyPath), nil
 }).(pulumi.StringOutput),
}, pulumi.DependsOn([]pulumi.Resource{
 renderPlaybookCmd,
 updatePythonCmd,
}))
if err != nil {
 return err
}

var playAnsiblePlaybookCmd = new Command.Local.Command("playAnsiblePlaybookCmd", new Command.Local.CommandArgs
{
 Create = wordpressEip.PublicIp.Apply(publicIp =>
 $"ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook "+
 $"-u ec2-user "+
 $"-i '{publicIp},' "+
 $"--private-key {privateKeyPath} "+
 $"playbook_rendered.yml"),
}, new CustomResourceOptions
{
 DependsOn =
 {
 renderPlaybookCmd,
 updatePythonCmd,
 },
});

 # Finally, play the Ansible playbook to finish installing.
 playAnsiblePlaybookCmd:
 type: command:local:Command
 properties:
 create: "ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook
 -u ec2-user
 -i '${wordpress-eip.publicIp},'
 --private-key ${privateKeyPath}
 playbook_rendered.yml"
 options:
 dependsOn:
 - ${renderPlaybookCmd}
 - ${updatePythonCmd}

That’s it! The entirety of your infrastructure — from the underlying AWS infrastructure, to the configuration commands to upgrade your server, to running the Ansible playbook which installs WordPress – is declared in a single Pulumi program in your language of choice. The only step left is to deploy it!

Provision and configure in one workflow

Combine Pulumi provisioning with configuration management tools like Ansible to build repeatable environments. Deploy your AWS infrastructure and configure your servers with a single pulumi up.

Get started

Seeing Everything in Action

To deploy this, you just need to do a few things.

You already created a project with pulumi new earlier, but can always create additional stacks using pulumi stack init.

Next, generate the key pair that will be used to access the EC2 instance over SSH:

$ ssh-keygen -f wordpress-keypair

This will create two files: your private (wordpress-keypair) and your public (wordpress-keypair.pub) keys. Keep your private key safe, since anybody with access to it can log into your EC2 machine!

Note that you may choose a different file name if you’re creating multiple stacks, for instance.

After that, simply set the Pulumi configuration values for the various settings. That includes any valid AWS region, the public and private keys, and a database password (which you should encrypt using --secret):

$ pulumi config set aws:region us-east-1 # any valid AWS region
$ pulumi config set publicKeyPath wordpress-keypair.pub # your newly generated public key
$ pulumi config set privateKeyPath wordpress-keypair # your newly generated private key
$ pulumi config set dbPassword Sup45ekreT@ --secret # your RDS database password -- keep it safe!

There are some other optional variables you can set if you choose. Feel free to skip these. If you don’t set them, they’ll receive the default values shown below:

$ pulumi config set dbInstanceSize db.t3.small # the RDS instance size to use
$ pulumi config set dbName wordpressdb # the name of the Wordpress database in RDS
$ pulumi config set dbUsername admin # the name of the Wordpress user that will be used
$ pulumi config set ec2InstanceSize t3.small # the EC2 instance size to provision

Now all you need to do is run pulumi up. This will do all of the magic, and you’ll see various things going on in the output: resources being created in AWS, commands being run locally and remotely, and the Ansible Playbook running and provisioning your Wordpress server:

$ pulumi up
Updating (dev)

 Type Name Status Info
 + pulumi:pulumi:Stack pulumi-ansible-wordpress-dev created
 + ├─ aws:ec2:Vpc prod-vpc created
 + ├─ aws:ec2:KeyPair wordpress-keypair created
 + ├─ aws:ec2:Subnet prod-subnet-private-1 created
 + ├─ aws:ec2:InternetGateway prod-igw created
 + ├─ aws:ec2:Subnet prod-subnet-private-2 created
 + ├─ aws:ec2:Subnet prod-subnet-public-1 created
 + ├─ aws:ec2:SecurityGroup ec2-allow-rule created
 + ├─ aws:ec2:RouteTable prod-public-rt created
 + ├─ aws:rds:SubnetGroup rds-subnet-grp created
 + ├─ aws:ec2:SecurityGroup rds-allow-rule created
 + ├─ aws:rds:Instance wordpressdb created
 + ├─ aws:ec2:RouteTableAssociation prod-rta-public-subnet-1 created
 + ├─ aws:ec2:Instance wordpress-instance created
 + ├─ command:local:Command renderPlaybookCmd created
 + ├─ aws:ec2:Eip wordpress-eip created
 + ├─ command:remote:Command updatePythonCmd created 12 messages
 + └─ command:local:Command playAnsiblePlaybookCmd created

Diagnostics:
 command:remote:Command (updatePythonCmd):
...

Outputs:
 url: "35.83.214.168"

Resources:
 + 18 created

Duration: 8m13s

After a few minutes, your new server will be ready! Notice that Pulumi orchestrates all of it, in the right order, including Ansible.

Now that everything’s up and running, let’s check out WordPress. The server’s security group allows HTTP(S) Internet traffic, so you just need to get its address. You probably noticed it was printed in the example output above. The program exports the resulting auto-allocated EIP address to make fetching it easy:

// Export the resulting wordpress EIP for easy access.
export const url = wordpressEip.publicIp;

# Export the resulting wordpress EIP for easy access.
pulumi.export("url", wordpress_eip.public_ip)

// Export the resulting wordpress EIP for easy access.
ctx.Export("url", wordpressEip.PublicIp)

// Export the resulting wordpress EIP for easy access.
this.Url = wordpressEip.PublicIp;

# Export the resulting wordpress EIP for easy access.
outputs:
 url: ${wordpress-eip.publicIp}

The IP address shows up at the end of pulumi up, but you can also access it with the pulumi stack output command:

$ pulumi stack output url
35.83.214.168

Using this, you can curl the WordPress server and see your handy work in action:

$ curl -L http://$(pulumi stack output url)
<!DOCTYPE html>
<html lang="en-US" xml:lang="en-US">
<head>...</head>
<body class="wp-core-ui language-chooser">
...

Or you can open a web browser to see a more visually pleasing WordPress welcome page:

$ open http://$(pulumi stack output url)

From there, feel free to experiment. You can simply make edits, rerun pulumi up, and it will incrementally update and deploy any changes you have made. After you’re done, you can destroy your stack and remove it, eliminating all AWS resources entirely:

$ pulumi destroy
$ pulumi stack rm

And there you go, cattle and pets, provisioning and configuration infrastructure as code, Pulumi and Ansible — better together!

Winding Down

In this article, you saw that provisioning and configuration don’t need to be an either-or. Indeed, they can be used together quite naturally. This is helpful for many configuration scenarios because infrastructure needs to be created before it can be configured, which is exactly what provisioning does. This can also be a great way to bridge between heritage workloads running on-prem or in hybrid VM environments, as you incrementally modernize your cloud architecture. Patching isn’t generally needed with containers because you just build and publish a new version, and roll out upgrades to it, and there’s a lot to like about that. But there’s no doubt, stateful servers are a reality that will be with us for the foreseeable future, and hopefully this article has given you some ideas how to incrementally modernize one step at a time.

The full code for this post is available on GitHub in all languages:

C#

Go

Python

TypeScript

YAML

Hierarchical Config: The Interim Solution

David Flanagan — Tue, 15 Mar 2022 10:00:00 +0000

This blog post discusses hierarchical config before Pulumi ESC was released. ESC makes hierarchical config easy.

A really common question that we receive on the Pulumi team is, “How can we set config at a project level, that can be used across all stacks?”.

When I say “really common” … I mean really, really common.

This issue was first open in 2018 and has received 52 votes from the community. Not only that, we’ve had plenty of similar issues created over the years too.

This is clearly a feature that our community has asked for! We’re happy to say that we delivered the first part of our plans to support hierarchical config in early November 2022. While we believe this new functionality satisfies most customer requests, below are some other approaches you can also use.

Project Level Config

Typically, when you need to access the stack configuration in a Pulumi program, you use the config package.

import { Config } from "@pulumi/pulumi";

const config = new Config();
const region = config.require("region");

import (
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi/config"
)

conf := config.New(ctx, "")
region := conf.Require("region")

import pulumi

config = pulumi.Config();
region = config.require('region');

using Pulumi;

var config = new Config();
var region = config.Require("region");

This code will reach from the Pulumi.<stack-name>.yaml file and give you access to the configuration values for the prefix you want to load.

So if you want project level config that works across all stacks, you can “hard code” the configuration. It sounds too simple, right?

export const projectLevelConfig = {
 region: "us-west-2",
 encryptionKmsKey: "arn:aws:kms:us-west-2:...",
 issueEmail: "bugs@ellingsonmineral.com"
};

type ProjectLevelConfig struct {
 Region string
 EncryptionKmsKey string
 IssueEmail string
}

const projectLevelConfig = ProjectLevelConfig{
 Region: "us-west-2",
 EncryptionKmsKey: "arn:aws:kms:us-west-2:...",
 IssueEmail: "bugs@ellingsonmineral.com"
}

projectLevelConfig = {
 "region": "us-west-2",
 "encryptionKmsKey": "arn:aws:kms:us-west-2:...",
 "issueEmail": "bugs@ellingsonmineral.com"
}

var projectLevelConfig = new Dictionary<string, string>
{
 { "region", "us-west-2" },
 { "encryptionKmsKey", "arn:aws:kms:us-west-2:..." },
 { "issueEmail", "bugs@ellingsonmineral.com"
}

These objects/structs/dictionaries can all be consumed and imported across the various components and files within your Pulumi program, but they cannot be used at an organizational level across multiple Pulumi programs … unless you’ve got a mono-repo. Do you? 😅

Organization Level Config

So if you want to provide organizational level config, you need to fall deeper into the programming language ecosystem that you’re using to build your infrastructure. Every programming language has a way to parse and consume JSON and YAML files. Better yet, every programming language has a way to fetch a remote JSON/YAML file and give you a object/struct/dictionary.

So really, there’s no reason you can’t use a remote JSON file to provide a common configuration object across all your Pulumi programs. That being said, there are definitely reasons you shouldn’t use a remote JSON file. Maintaining a remote JSON file is much more of a burden than it will initially seem. As you add more values to your JSON file and more teams/projects begin to rely on it, you’ll start to feel the pain of schema management. How do I know that the JSON I’m pulling down has the values I need and that fields haven’t been changed or replaced? 😬

So while I 💯 feel like you shouldn’t do this, you can if you really need to. Just make sure you understand the tradeoffs and enforce a schema.

A common way to manage schema is to use a JSON Schema or CUE to define the structure of your JSON file.

By using one of these methods, you can publish a schema that is available within your organization and people can have confidence that the value they pull remotely can be deserialized to a strict type. Using CI/CD you can also ensure the value itself conforms to the schema before updating the public document.

With JSON Schema, you’d define something like:

{
 "type": "object",
 "properties": {
 "region": {
 "type": "string",
 "default": "us-west-2"
 },
 "encryptionKmsKey": {
 "type": "string",
 "default": "arn:aws:kms:us-west-2:..."
 },
 "issueEmail": {
 "type": "string",
 "default": "bugs@ellingsonmineral.com"
 }
 }
}

With CUE, this is much more concise. CUE is really awesome and I encourage you to check it out.

region: string | *"eu-west-2"
encryptionKmsKey: string | *"arn:aws:kms:eu-west-2:..."
issueEmail: string | *"bugs@ellingsonmineral.com"

Pulumi Native Hierarchical Configuration

So we’ve worked out that project level configuration is a data structure within our Pulumi programs and it works pretty darn well.

However, the organization level config is prone and rife with pain, confusion, and delusion. It requires non-trivial tooling and process to ensure that a global document is valid and consistent with downstream consumers. What I’ve shown you is a “if you must” approach that I would use myself. We’ve also not even handled the “merge” of organization level with project level overrides, and then stack level overrides. Of course, with code it’s all possible, but it’s more and more code that you need to write that we don’t want you to need to write, right? Right!

Pulumi wants to make this better for you, which is why we added project-level config support and why we are actively exploring additional enhancements. If you want to follow along with our progress on improving the configuration experience, I suggest you watch/subscribe to this issue.

If you want to share your opinions on how YOU would like this feature to be implemented, jump into the comments. We’d love to hear from you!

Speak soon.

Pulumi Blog: Configuration

The Pulumi 'Push to start' GitOps Experience

Index

New technology learning advice

Tech learning hacks:

A DevOps journey

DevOps day 0

What’s in a DevOps day 0?

Prerequisites

Commandline Utilities:

Cloud-native … cloud developer environment?

DevOps day 1

~$ pulumi new --ai

Step 1. Login to Pulumi Cloud

Step 2. Create a new program

Step 3. YOLO for fun and profit!

DevOps day 2

Conclusion

Did it deploy?

What’s in the recipe?

Git Code Repository

Direnv: Automatic Environment Variables

Pulumi Devcontainer

Pulumi Cloud

KinD: Kubernetes-in-docker

Now let’s create the deployment code!

Cleanup

Simplify Configuration Management with Project-level Config

How to Use Project-level Configuration

Defining Configuration Schemas in Pulumi

Pulumi YAML Configuration: What’s Changing?

What’s Next for Pulumi Configuration?

FAQ: Project-Level Configuration in Pulumi

What is project-level config in Pulumi?

Can I override project-level config for specific stacks?

How does Pulumi handle object values in config?

Is the configuration key still supported in YAML?

Where can I give feedback or ask questions?

Introducing the Azure Static Website Component

Azure Static Website Component

Using the component to deploy a static website

Create an AWS Static Website Fast with Angular and Pulumi

What is Angular?

What are Components in Angular?

What is the AWS Static Website Component?

The Angular Components

Building the Pulumi Café Website

Deploy the Site with Pulumi

Modify the .yaml File

Manage Your Site with Pulumi

Learn More

Deploy Static Websites to AWS using 10 lines of YAML

What the Component Does

Some Examples

Create a Website

Get Started

Deploy the Website

What You Gain

Input Properties

Output Properties

Add a CloudFront CDN

More Pulumi Advantages

Learn More

Install Pulumi

Shared configuration stacks with AWS Systems Manager

Sketching it out

Building the shared config stack

Fetching configuration at deploy-time

Fetching configuration at runtime

Wrapping up, and next steps

Deploy WordPress to AWS using Pulumi and Ansible

Preamble: On Provisioning and Configuration

Pulumi and Ansible Better Together - The Approach

Creating Your Infrastructure as Code Project

Provisioning the AWS Network

Provisioning the AWS EC2 Instance and RDS Database

Configuring the AWS EC2 Instance Using the Pulumi Command Package

Configuring WordPress on EC2 Using Ansible

Seeing Everything in Action

Winding Down

`~$ pulumi new --ai`