Pulumi Blog: Zack Chase

Introducing the Community AWS IAM Package

Zack Chase — Wed, 05 Oct 2022 09:24:26 -0700

Building on top of the cloud can be frustrating at times. You will likely run into resources that complicated to create, others that are very tedious to create, and worst of all resources that are complicated and tedious to create. As cloud engineers ourselves, we feel the same pain as our users and strive to build abstractions that make cloud engineering a more productive and pleasant experience.

Recently we released our Community AWS IAM Package to help deliver on the promise of making the cloud easier to use for every operator, engineer, and user. This package is based on the Terraform AWS IAM Module, so it allows our users to take advantage of battle-tested abstractions. The package also helps transitioning Terraform users by maintaining similar resource names and inputs so they can focus on taking advantage of features of their programming language of choice (TypeScript, JavaScript, Python, Go, .NET, and YAML).

Examples

In the following sections, we will show a few examples of common use cases when using the Community AWS IAM Package.

Assumable Role with SAML

The example code below shows how you can create an Assumable Role with SAML, which will create a single IAM role which can be assumed by users with a SAML Identity Provider.

export const assumableRoleWithSaml = new iam.AssumableRoleWithSAML("aws-iam-example-assumable-role-with-saml", {
 providerIds: [ "arn:aws:iam::235367859851:saml-provider/idp_saml" ],
 role: {
 name: "saml-role",
 policyArns: [ "arn:aws:iam::aws:policy/ReadOnlyAccess" ],
 },
 tags: {
 Role: "saml-role",
 },
});

assumable_role_with_saml = iam.AssumableRoleWithSAML(
 'assumable_role_with_saml',
 role=iam.RoleArgs(
 name='saml-role',
 policy_arns=['arn:aws:iam::aws:policy/ReadOnlyAccess'],
 ),
 tags={
 'Role': 'saml-role',
 },
 provider_ids=['arn:aws:iam::235367859851:saml-provider/idp_saml']
)

assumableRoleWithSAML, err := iam.NewAssumableRoleWithSAML(ctx, "assumable-role-with-saml", &iam.AssumableRoleWithSAMLArgs{
 Role: iam.RoleArgs{
 Name: pulumi.String("saml-role"),
 PolicyArns: pulumi.ToStringArray([]string{"arn:aws:iam::aws:policy/ReadOnlyAccess"}),
 },
 Tags: pulumi.ToStringMap(map[string]string{
 "Role": "saml-role",
 }),
 ProviderIds: pulumi.ToStringArray([]string{"arn:aws:iam::235367859851:saml-provider/idp_saml"}),
})
if err != nil {
 return err
}

var assumableRoleWithSaml = new AssumableRoleWithSAML("assumable-role-with-saml", new AssumableRoleWithSAMLArgs
{
 Role = new RoleArgs
 {
 Name = "saml-role",
 PolicyArns = {"arn:aws:iam::aws:policy/ReadOnlyAccess"},
 },
 Tags = new InputMap<string>
 {
 {"Role", "saml-role"},
 },
 ProviderIds = {"arn:aws:iam::235367859851:saml-provider/idp_saml"},
});

resources:
 assumableRoleWithSaml:
 type: "aws-iam:index:AssumableRoleWithSAML"
 properties:
 role:
 name: "saml-role"
 policyArns:
 - "arn:aws:iam::aws:policy/ReadOnlyAccess"
 tags:
 Role: "saml-role"
 providerIds:
 - "arn:aws:iam::235367859851:saml-provider/idp_saml"

EKS Assumable Role

The example code below shows how you can create an EKS Assumable Role, which will create an IAM Role that can be assumed by multiple EKS Service Accounts.

export const eksRole = new iam.EKSRole("aws-iam-example-eks-role", {
 role: {
 name: "eks-role",
 policyArns: [ "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy" ],
 },
 tags: {
 Name: "eks-role",
 },
 clusterServiceAccounts: {
 "cluster1": [ "default:my-app" ],
 "cluster2": [ "default:my-app", "canary:my-app" ],
 },
});

eks_role = iam.EKSRole(
 'eks_role',
 role=iam.RoleArgs(
 name='eks-role',
 policy_arns=['arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy'],
 ),
 tags={
 'Name': 'eks-role',
 },
 cluster_service_accounts={
 'cluster1': [ 'default:my-app' ],
 'cluster2': [ 'default:my-app', 'canary:my-app' ],
 },
)

eksRole, err := iam.NewEKSRole(ctx, "eks-role", &iam.EKSRoleArgs{
 Role: iam.RoleArgs{
 Name: pulumi.String("eks-role"),
 PolicyArns: pulumi.ToStringArray([]string{"arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"}),
 },
 Tags: pulumi.ToStringMap(map[string]string{
 "Role": "eks-role",
 }),
 ClusterServiceAccounts: pulumi.ToStringArrayMap(map[string][]string{
 "cluster1": {"default:my-app"},
 "cluster2": {"default:my-app", "canary:my-app"},
 }),
})
if err != nil {
 return err
}

var eksRole = new EKSRole("eks-role", new EKSRoleArgs
{
 Role = new RoleArgs
 {
 Name = "eks-role",
 PolicyArns = {"arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"},
 },
 Tags = new InputMap<string>
 {
 {"Name", "eks-role"},
 },
 ClusterServiceAccounts = {
 {"cluster1", ImmutableArray.Create<string>(new string[] {"default:my-app"})},
 {"cluster2", ImmutableArray.Create<string>(new string[] {"default:my-app", "canary:my-app"})}
 },
});

resources:
 eksRole:
 type: "aws-iam:index:EKSRole"
 properties:
 role:
 name: "eks-role"
 policyArns:
 - "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
 tags:
 Name: "eks-role"
 clusterServiceAccounts:
 cluster1:
 - "default:my-app"
 cluster2:
 - "default:my-app"
 - "canary:my-app"

IAM Role for an EKS Service Account

The example code below shows how you can create an IAM Role for an EKS Service Account, which will create an IAM Role with various Policies attached that can be assumed by multiple EKS Service Accounts.

export const roleForServiceAccountsEks = new iam.RoleForServiceAccountsEks("aws-iam-example-role-for-service-accounts-eks", {
 role: {
 name: "vpc-cni"
 },
 tags: {
 Name: "vpc-cni-irsa",
 },
 oidcProviders: {
 main: {
 providerArn: "arn:aws:iam::012345678901:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/5C54DDF35ER19312844C7333374CC09D",
 namespaceServiceAccounts: ["default:my-app", "canary:my-app"],
 }
 },
 policies: {
 vpnCni: {
 attach: true,
 enableIpv4: true,
 },
 },
});

role_for_service_account_eks = iam.RoleForServiceAccountsEks(
 'role_for_service_account_eks',
 role=iam.RoleArgs(
 name='vpc-cni'
 ),
 tags={
 'Name': 'vpc-cni-irsa',
 },
 oidc_providers={
 'main': iam.OIDCProviderArgs(
 provider_arn='arn:aws:iam::012345678901:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/5C54DDF35ER19312844C7333374CC09D',
 namespace_service_accounts=['default:my-app', 'canary:my-app'],
 ),
 },
 policies=iam.EKSRolePoliciesArgs(
 vpn_cni=iam.EKSVPNCNIPolicyArgs(
 attach=True,
 enable_ipv4=True,
 ),
 ),
)

roleForServiceAccountsEKS, err := iam.NewRoleForServiceAccountsEks(ctx, "role-for-service-accounts-eks", &iam.RoleForServiceAccountsEksArgs{
 Role: iam.EKSServiceAccountRolePtr(&iam.EKSServiceAccountRoleArgs{
 Name: pulumi.String("vpc-cni"),
 }),
 Tags: pulumi.ToStringMap(map[string]string{
 "Name": "vpc-cni-irsa",
 }),
 OidcProviders: iam.OIDCProviderMap{
 "main": iam.OIDCProviderArgs{
 ProviderArn: pulumi.String("arn:aws:iam::012345678901:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/5C54DDF35ER19312844C7333374CC09D"),
 NamespaceServiceAccounts: pulumi.ToStringArray([]string{"default:my-app", "canary:my-app"}),
 },
 },
 Policies: iam.EKSRolePoliciesPtr(&iam.EKSRolePoliciesArgs{
 VpnCni: iam.EKSVPNCNIPolicyPtr(&iam.EKSVPNCNIPolicyArgs{
 Attach: pulumi.Bool(true),
 EnableIpv4: pulumi.BoolPtr(true),
 }),
 }),
})
if err != nil {
 return err
}

var roleForServiceAccountEks = new RoleForServiceAccountsEks("role-for-service-account-eks", new RoleForServiceAccountsEksArgs
{
 Role = new EKSServiceAccountRoleArgs
 {
 Name = "vpn-cni",
 },
 Tags = {
 {"Name", "vpc-cni-irsa"},
 },
 OidcProviders = {
 {"main", new OIDCProviderArgs
 {
 ProviderArn = "arn:aws:iam::012345678901:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/5C54DDF35ER19312844C7333374CC09D",
 NamespaceServiceAccounts = {"default:my-app", "canary:my-app"},
 }},
 },
 Policies = new EKSRolePoliciesArgs
 {
 VpnCni = new EKSVPNCNIPolicyArgs
 {
 Attach = true,
 EnableIpv4 = true,
 },
 },
});

resources:
 roleForServiceAccountsEks:
 type: "aws-iam:index:RoleForServiceAccountsEks"
 properties:
 role:
 name: "vpc-cni"
 tags:
 Name: "vpc-cni-irsa"
 oidcProviders:
 main:
 providerArn: "arn:aws:iam::012345678901:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/5C54DDF35ER19312844C7333374CC09D"
 namespaceServiceAccounts:
 - "default:my-app"
 - "canary:my-app"
 policies:
 vpnCni:
 attach: true
 enableIpv4: true

You can find a complete list of supported EKS Policies within the API Documentation.

More Use Cases

A few of the other use cases support by the Community AWS IAM package are:

Learn More

If you want to learn more about Pulumi and the Community AWS IAM package, here are a few links you might find interesting:

Read the AWS IAM documentation to learn more about the package, how to install and configure it and use its APIs.
If you’re interested in learning more about Pulumi concepts, try the Concepts page.

I'll just update the bucket object. What could go wrong?

Zack Chase — Mon, 12 Sep 2022 08:33:30 -0700

Creating a website on AWS with an S3 bucket is a fairly straightforward task. You just need to create an S3 bucket, configure it to be a website, and add your content, right?

Unfortunately getting your content into a bucket is only a part of the story. To get your website ready to handle traffic, you will want to associate a domain name and likely want to use a CDN, like Cloudfront, to help with performance. The AWS Static Website Package makes it easy for you to associate a domain and stand up a CDN with only a few arguments.

name: atomic-bucket-blog-yaml
description: An AWS Pulumi YAML program for provisioning a static website.
runtime: yaml
resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "./build"
outputs:
 websiteURL: ${web.websiteURL}

import * as pulumi from "@pulumi/pulumi";
import * as awsStaticWebsite from "@pulumi/aws-static-website";

const web = new awsStaticWebsite.Website("web", {
 sitePath: "./build"
});

export const websiteURL = web.websiteURL;

import pulumi
import pulumi_aws_static_website as aws_static_website

web = aws_static_website.Website("web", site_path="./build")

pulumi.export("websiteURL", web.website_url)

package main

import (
 website "github.com/pulumi/pulumi-aws-static-website/sdk/go/aws-static-website"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 web, err := website.NewWebsite(ctx, "web", &website.WebsiteArgs{
 SitePath: pulumi.String("./build"),
 })
 if err != nil {
 return err
 }
 ctx.Export("websiteURL", web.WebsiteURL)
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using AwsStaticWebsite = Pulumi.AwsStaticWebsite;

return await Deployment.RunAsync(() =>
{
 var web = new AwsStaticWebsite.Website("web", new()
 {
 SitePath = "./build",
 });

 return new Dictionary<string, object?>
 {
 ["websiteURL"] = web.WebsiteURL,
 };
});

The only problem? Now you have to deal with keeping your content up to date without interrupting your website’s user experience.

So what’s the problem?

When your website is still fairly small in size, updating the bucket objects in place probably seems like a decent strategy for updating your static website. Unfortunately updating the objects in place has a few rough edges that could lead to a degraded user experience.

First and foremost (in my opinion) is that you are introducing a chance for assets to not be available when the page needs them. For example, if an HTML page is updated with new CSS classes before the CSS itself is updated, then the website could have bad styling until the new CSS file is available.

Having a CDN to cache your content is helpful in mitigating deploy time issues, but it doesn’t eliminate them entirely. Different objects can have different expiration times, so you still risk an asset not being available if a cached paged is requesting an object that has expired.

Versioning your website

To eliminate all the potential issues that could go wrong with an in-place update, you can version your website to ensure all your assets and dependencies are present and the cache behaves in the way you intend. While that may seem complicated, the AWS Static Website Package makes it simple for you to enable website versioning.

Atomic Deployments

The first step for versioning is that you are going to need to build a new S3 Bucket on every deployment. You want to do this because it ensures all the assets and dependencies are present in the bucket before you associate it with your Cloudfront CDN.

To accomplish this with the AWS Static Website Package all you need to do is set the Atomic Deployment argument to true.

name: atomic-bucket-blog-yaml
description: An AWS Pulumi YAML program for provisioning a static website.
runtime: yaml
resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "./build"
 atomicDeployments: true
outputs:
 websiteURL: ${web.websiteURL}

import * as pulumi from "@pulumi/pulumi";
import * as awsStaticWebsite from "@pulumi/aws-static-website";

const web = new awsStaticWebsite.Website("web", {
 sitePath: "./build",
 atomicDeployments: true,
});

export const websiteURL = web.websiteURL;

import pulumi
import pulumi_aws_static_website as aws_static_website

web = aws_static_website.Website("web",
 site_path="./build",
 atomic_deployments=True)

pulumi.export("websiteURL", web.website_url)

package main

import (
 website "github.com/pulumi/pulumi-aws-static-website/sdk/go/aws-static-website"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 web, err := website.NewWebsite(ctx, "web", &website.WebsiteArgs{
 SitePath: pulumi.String("./build"),
 AtomicDeployments: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }
 ctx.Export("websiteURL", web.WebsiteURL)
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using AwsStaticWebsite = Pulumi.AwsStaticWebsite;

return await Deployment.RunAsync(() =>
{
 var web = new AwsStaticWebsite.Website("web", new()
 {
 SitePath = "./build",
 AtomicDeployments = true,
 });

 return new Dictionary<string, object?>
 {
 ["websiteURL"] = web.WebsiteURL,
 };
});

With atomic deployments enabled a new bucket will be provisioned, your content is added via a sync command, and finally once your new bucket is built the old bucket is deleted.

Rollback

The AWS Static Website Package currently does not support rollbacks but if you export the name of the bucket on your stack then the Package will keep the previous bucket provision until the next update. This allows manual rollbacks to previous bucket versions if you encounter a post-deployment error. See the example code below for how to enable this feature exactly.

name: atomic-bucket-blog-yaml
description: An AWS Pulumi YAML program for provisioning a static website.
runtime: yaml
resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "./build"
 atomicDeployments: true
outputs:
 websiteURL: ${web.websiteURL}
 bucketName: ${web.bucketName}

import * as pulumi from "@pulumi/pulumi";
import * as aws_static_website from "@pulumi/aws-static-website";

const web = new aws_static_website.Website("web", {
 sitePath: "./build",
 atomicDeployments: true,
});

export const websiteURL = web.websiteURL;
export const bucketName = web.bucketName;

import pulumi
import pulumi_aws_static_website as aws_static_website

web = aws_static_website.Website("web",
 site_path="./build",
 atomic_deployments=True)

pulumi.export("websiteURL", web.website_url)
pulumi.export("bucketName", web.bucket_name)

package main

import (
 website "github.com/pulumi/pulumi-aws-static-website/sdk/go/aws-static-website"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 web, err := website.NewWebsite(ctx, "web", &website.WebsiteArgs{
 SitePath: pulumi.String("./build"),
 AtomicDeployments: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }
 ctx.Export("websiteURL", web.WebsiteURL)
 ctx.Export("bucketName", web.BucketName)
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using AwsStaticWebsite = Pulumi.AwsStaticWebsite;

return await Deployment.RunAsync(() =>
{
 var web = new AwsStaticWebsite.Website("web", new()
 {
 SitePath = "./build",
 AtomicDeployments = true,
 });

 return new Dictionary<string, object?>
 {
 ["websiteURL"] = web.WebsiteURL,
 ["bucketName"] = web.BucketName,
 };
});

Cache Control

Now that your website has atomic deployments enabled and you have solved the issue where your assets and dependencies might not be available, you still have to tackle the beast that is your CDN cache. You will need a way to tell the CDN that there is new content and to stop serving the old content.

You could just rely on the cached content expiring but as stated earlier that can still lead in some cases to missing assets. The AWS Static Website Package handles communicating to the CDN that there is new content by provisioning a Cloudfront Function that adds a unique build identifier to the CDN’s cache key.

function handler(event){
 var request = event.request;
 request.headers["website-version"] = {
 value: "${buildIdentifier}",
 };
 return request;
}

To enable this behavior you simply need to set the cache control header argument to true.

name: atomic-bucket-blog-yaml
description: An AWS Pulumi YAML program for provisioning a static website.
runtime: yaml
resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "./build"
 withCDN: true
 addWebsiteVersionHeader: true
outputs:
 websiteURL: ${web.websiteURL}

import * as pulumi from "@pulumi/pulumi";
import * as aws_static_website from "@pulumi/aws-static-website";

const web = new aws_static_website.Website("web", {
 sitePath: "./build",
 withCDN: true,
 addWebsiteVersionHeader: true,
});

export const websiteURL = web.websiteURL;

import pulumi
import pulumi_aws_static_website as aws_static_website

web = aws_static_website.Website("web",
 site_path="./build",
 with_cdn=True,
 add_website_version_header=True)

pulumi.export("websiteURL", web.website_url)

package main

import (
 website "github.com/pulumi/pulumi-aws-static-website/sdk/go/aws-static-website"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 web, err := website.NewWebsite(ctx, "web", &website.WebsiteArgs{
 SitePath: pulumi.String("./build"),
 WithCDN: pulumi.Bool(true),
 AddWebsiteVersionHeader: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }
 ctx.Export("websiteURL", web.WebsiteURL)
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using AwsStaticWebsite = Pulumi.AwsStaticWebsite;

return await Deployment.RunAsync(() =>
{
 var web = new AwsStaticWebsite.Website("web", new()
 {
 SitePath = "./build",
 WithCDN = true,
 AddWebsiteVersionHeader = true,
 });

 return new Dictionary<string, object?>
 {
 ["websiteURL"] = web.WebsiteURL,
 };
});

Putting it all together

Below you will find a full Pulumi program that atomically deploys your website, keeps the cache up to date, and allows for you to do manual rollbacks.

name: atomic-bucket-blog-yaml
description: An AWS Pulumi YAML program for provisioning a static website.
runtime: yaml
resources:
 web:
 type: "aws-static-website:index:Website"
 properties:
 sitePath: "./build"
 withCDN: true
 atomicDeployments: true
 addWebsiteVersionHeader: true
outputs:
 websiteURL: ${web.websiteURL}
 bucketName: ${web.bucketName}

import * as pulumi from "@pulumi/pulumi";
import * as aws_static_website from "@pulumi/aws-static-website";

const web = new aws_static_website.Website("web", {
 sitePath: "./build",
 withCDN: true,
 atomicDeployments: true,
 addWebsiteVersionHeader: true,
});

export const websiteURL = web.websiteURL;
export const bucketName = web.bucketName;

import pulumi
import pulumi_aws_static_website as aws_static_website

web = aws_static_website.Website("web",
 site_path="./build",
 with_cdn=True,
 atomic_deployments=True,
 add_website_version_header=True)

pulumi.export("websiteURL", web.website_url)
pulumi.export("bucketName", web.bucket_name)

package main

import (
 website "github.com/pulumi/pulumi-aws-static-website/sdk/go/aws-static-website"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 web, err := website.NewWebsite(ctx, "web", &website.WebsiteArgs{
 SitePath: pulumi.String("./build"),
 WithCDN: pulumi.bool(true),
 AtomicDeployments: pulumi.Bool(true),
 AddWebsiteVersionHeader: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }
 ctx.Export("websiteURL", web.WebsiteURL)
 ctx.Export("bucketName", web.BucketName)
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using AwsStaticWebsite = Pulumi.AwsStaticWebsite;

return await Deployment.RunAsync(() =>
{
 var web = new AwsStaticWebsite.Website("web", new()
 {
 SitePath = "./build",
 WithCDN = true,
 AtomicDeployments = true,
 AddWebsiteVersionHeader = true,
 });

 return new Dictionary<string, object?>
 {
 ["websiteURL"] = web.WebsiteURL,
 ["bucketName"] = web.BucketName,
 };
});

Now when you run a pulumi up you will provision a new bucket, sync your content, associate the bucket to your CDN, update the Cloudfront Function so the CDN knows to fetch new content, and lastly keep your previous bucket around in case you need to rollback.

Deploying and managing a static website should be easy and the AWS Static Website Package makes it so you can focus more time on generating great content and less time on the internals on your website. For more information on the AWS Static Website Package you can visit the documentation or via the code directly on Github.

Unlocking Your Data With Metabase and AWS Fargate

Zack Chase — Tue, 09 Aug 2022 08:57:51 -0700

I love data. I mean, I really love data. Data gives you the ability to understand the world around you and, to a certain degree, project what the future could look like. At Pulumi we use data every day to help make smarter product and business decisions.

Though one hurdle we encountered was not only the sheer volume of data we have but also the large disparity of systems storing that data. Like many companies before us, we chose to build a data warehouse, specifically AWS Redshift, to store all of our data.

After grinding through the work to consolidate our data into a single source of truth, we felt like we had conquered the world. That feeling was fleeting as we quickly realized we needed a scalable way to save, abstract, and collaborate with our data across the entire organization.

Using Metabase to unlock our data

As an early stage start-up at the time, we needed a low-cost solution (free) which limited our available options. One advantage we did have at the time was that as a cloud native start-up we were comfortable with the upfront investment and operational overhead of self-hosting the solution. With all of our parameters set, we eventually settled on using Metabase.

What is Metabase?

Metabase is a Business Intelligence Tool that enables you to visualize and collaborate using data from a variety of databases. You do not need to understand the underlying database’s query language, SQL for example, because Metabase provides a UI to interact with your data. You can however drop into a query editor at any moment to perform more advanced queries.

Metabase provides you with two options to run Metabase in production, Open Source (self-managed) and Enterprise (managed). The main difference between the two is that with Open Source you have to manage the infrastructure yourself, whereas with Enterprise, Metabase will manage the infrastructure for you for a fee.

Deploying Metabase to production

Once we settled on our business intelligence tool it was time to wire it up and start living that sweet data-driven lifestyle. To run the service yourself, Metabase provides you with two options: a Docker Image or a .JAR. We were already running our SaaS service on ECS, so naturally, we opted to go with the Docker approach. In addition to running the Docker Image, we also needed to provision a database so that Metabase could store the relevant data it needs to run the service.

To accomplish the task of provisioning all the required resources, the team wrote a Pulumi Component to encapsulate all the complexity and provide a simple interface for provisioning and updating the service. Unfortunately, this was before we had developed Pulumi Packages and the Component was only available within the program itself.

The Metabase Package

That brings us to today, and we are happy to deliver an open source Pulumi Package for running Metabase on AWS ECS. With Pulumi’s Metabase Package, you can quickly get started with Metabase without having to worry about the underlying infrastructure.

In the below sections we will look at example configurations of the Metabase resource, all the way from a bare bones configuration to a more complex configuration with a custom VPC, subnets, and domain.

Default

By default, the Package accepts zero arguments and will deploy your resources into your default VPC, on public subnets, and without a custom domain name.

import * as pulumi from "@pulumi/pulumi";
import * as metabase from "@pulumi/metabase";

const metabaseService = new metabase.Metabase("metabaseService", {});

export const url = metabaseService.dnsName;

import pulumi
import pulumi_metabase as metabase

metabase_service = metabase.Metabase("metabaseService")

pulumi.export("url", metabase_service.dns_name)

package main

import (
 "github.com/pulumi/pulumi-metabase/sdk/go/metabase"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 metabaseService, err := metabase.NewMetabase(ctx, "metabaseService", nil)
 if err != nil {
 return err
 }
 ctx.Export("url", metabaseService.DnsName)
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Metabase = Pulumi.Metabase;

return await Deployment.RunAsync(() =>
{
 var metabaseService = new Metabase.Metabase("metabaseService");

 return new Dictionary<string, object?>
 {
 ["url"] = metabaseService.DnsName,
 };
});

name: metabase-yaml
runtime: yaml
resources:
 metabaseService:
 type: "metabase:index:Metabase"
outputs:
 url: ${metabaseService.dnsName}

Custom VPC

If you would like to run the service in a specific VPC, then you can provide a VPC ID as an argument. The example below will provision your resources in your defined VPC, on public subnets, and without a custom domain.

import * as pulumi from "@pulumi/pulumi";
import * as metabase from "@pulumi/metabase";

const metabaseService = new metabase.Metabase("metabaseService", {
 vpcId: "vpc-123",
});

export const url = metabaseService.dnsName;

import pulumi
import pulumi_metabase as metabase

metabase_service = metabase.Metabase("metabaseService",
 vpc_id="vpc-123")

pulumi.export("url", metabase_service.dns_name)

package main

import (
 "github.com/pulumi/pulumi-metabase/sdk/go/metabase"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 metabaseService, err := metabase.NewMetabase(ctx, "metabaseService", &metabase.MetabaseArgs{
 VpcId: pulumi.String("vpc-123"),
 })
 if err != nil {
 return err
 }
 ctx.Export("url", metabaseService.DnsName)
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Metabase = Pulumi.Metabase;

return await Deployment.RunAsync(() =>
{
 var metabaseService = new Metabase.Metabase("metabaseService", new()
 {
 VpcId = "vpc-123",
 });

 return new Dictionary<string, object?>
 {
 ["url"] = metabaseService.DnsName,
 };
});

name: metabase-yaml
runtime: yaml
resources:
 metabaseService:
 type: "metabase:index:Metabase"
 properties:
 vpcId: "vpc-123"
outputs:
 url: ${metabaseService.dnsName}

Custom Networking

If you would like to run the Load Balancer, Database, or ECS Service on specific subnets you can provide the IDs of those subnets. The subnets you provide need to be a part of the VPC you are deploying into. The example below will provision your resources in your defined VPC (or default if you do not provide a VPC ID), on your specified subnets, and without a custom domain.

import * as pulumi from "@pulumi/pulumi";
import * as metabase from "@pulumi/metabase";

const metabaseService = new metabase.Metabase("metabaseService", {
 vpcId: "vpc-123",
 networking: {
 ecsSubnetIds: [
 "subnet-123",
 "subnet-456",
 ],
 dbSubnetIds: [
 "subnet-789",
 "subnet-abc",
 ],
 lbSubnetIds: [
 "subnet-def",
 "subnet-ghi",
 ],
 },
});

export const url = metabaseService.dnsName;

import pulumi
import pulumi_metabase as metabase

metabase_service = metabase.Metabase("metabaseService",
 vpc_id="vpc-123",
 networking=metabase.NetworkingArgs(
 ecs_subnet_ids=[
 "subnet-123",
 "subnet-456",
 ],
 db_subnet_ids=[
 "subnet-789",
 "subnet-abc",
 ],
 lb_subnet_ids=[
 "subnet-def",
 "subnet-ghi",
 ],
 ))

pulumi.export("url", metabase_service.dns_name)

package main

import (
 "github.com/pulumi/pulumi-metabase/sdk/go/metabase"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 metabaseService, err := metabase.NewMetabase(ctx, "metabaseService", &metabase.MetabaseArgs{
 VpcId: pulumi.String("vpc-123"),
 Networking: &metabase.NetworkingArgs{
 EcsSubnetIds: pulumi.StringArray{
 pulumi.String("subnet-123"),
 pulumi.String("subnet-456"),
 },
 DbSubnetIds: pulumi.StringArray{
 pulumi.String("subnet-789"),
 pulumi.String("subnet-abc"),
 },
 LbSubnetIds: pulumi.StringArray{
 pulumi.String("subnet-def"),
 pulumi.String("subnet-ghi"),
 },
 },
 })
 if err != nil {
 return err
 }
 ctx.Export("url", metabaseService.DnsName)
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Metabase = Pulumi.Metabase;

return await Deployment.RunAsync(() =>
{
 var metabaseService = new Metabase.Metabase("metabaseService", new()
 {
 VpcId = "vpc-123",
 Networking = new Metabase.Inputs.NetworkingArgs
 {
 EcsSubnetIds = new[]
 {
 "subnet-123",
 "subnet-456",
 },
 DbSubnetIds = new[]
 {
 "subnet-789",
 "subnet-abc",
 },
 LbSubnetIds = new[]
 {
 "subnet-def",
 "subnet-ghi",
 },
 },
 });

 return new Dictionary<string, object?>
 {
 ["url"] = metabaseService.DnsName,
 };
});

name: metabase-yaml
runtime: yaml
resources:
 metabaseService:
 type: "metabase:index:Metabase"
 properties:
 vpcId: "vpc-123"
 networking:
 ecsSubnetIds: [ "subnet-123", "subnet-456" ]
 dbSubnetIds: [ "subnet-789", "subnet-abc" ]
 lbSubnetIds: [ "subnet-def", "subnet-ghi" ]
outputs:
 url: ${metabaseService.dnsName}

Custom Domain

If you would like to have your service run behind a custom domain, you can specify the hosted zone and domain name as arguments. The example below will provision your resources in your default VPC, on public subnets, with a custom domain name.

import * as pulumi from "@pulumi/pulumi";
import * as metabase from "@pulumi/metabase";

const metabaseService = new metabase.Metabase("metabaseService", {
 domain: {
 hostedZoneName: "example.com",
 domainName: "metabase.example.com",
 },
});

export const url = metabaseService.dnsName;

import pulumi
import pulumi_metabase as metabase

metabase_service = metabase.Metabase("metabaseService",
 domain=metabase.CustomDomainArgs(
 hosted_zone_name="example.com",
 domain_name="metabase.example.com",
 ))

pulumi.export("url", metabase_service.dns_name)

package main

import (
 "github.com/pulumi/pulumi-metabase/sdk/go/metabase"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 metabaseService, err := metabase.NewMetabase(ctx, "metabaseService", &metabase.MetabaseArgs{
 Domain: &metabase.CustomDomainArgs{
 HostedZoneName: pulumi.String("example.com"),
 DomainName: pulumi.String("metabase.example.com"),
 },
 })
 if err != nil {
 return err
 }
 ctx.Export("url", metabaseService.DnsName)
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Metabase = Pulumi.Metabase;

return await Deployment.RunAsync(() =>
{
 var metabaseService = new Metabase.Metabase("metabaseService", new()
 {
 VpcId = "vpc-123",
 Domain = new Metabase.Inputs.CustomDomainArgs
 {
 HostedZoneName = "example.com",
 DomainName = "metabase.example.com",
 },
 });

 return new Dictionary<string, object?>
 {
 ["url"] = metabaseService.DnsName,
 };
});

name: metabase-yaml
runtime: yaml
resources:
 metabaseService:
 type: "metabase:index:Metabase"
 properties:
 vpcId: "vpc-123"
 domain:
 hostedZoneName: "example.com"
 domainName: "metabase.example.com"
outputs:
 url: ${metabaseService.dnsName}

All Together Now

You can provide all the arguments at once if you would like more control over your infrastructure. The below example will provision your resources in your defined VPC, with your defined subnets, and have a custom domain.

import * as pulumi from "@pulumi/pulumi";
import * as metabase from "@pulumi/metabase";

const metabaseService = new metabase.Metabase("metabaseService", {
 vpcId: "vpc-123",
 networking: {
 ecsSubnetIds: [
 "subnet-123",
 "subnet-456",
 ],
 dbSubnetIds: [
 "subnet-789",
 "subnet-abc",
 ],
 lbSubnetIds: [
 "subnet-def",
 "subnet-ghi",
 ],
 },
 domain: {
 hostedZoneName: "example.com",
 domainName: "metabase.example.com",
 },
});

export const url = metabaseService.dnsName;

import pulumi
import pulumi_metabase as metabase

metabase_service = metabase.Metabase("metabaseService",
 vpc_id="vpc-123",
 networking=metabase.NetworkingArgs(
 ecs_subnet_ids=[
 "subnet-123",
 "subnet-456",
 ],
 db_subnet_ids=[
 "subnet-789",
 "subnet-abc",
 ],
 lb_subnet_ids=[
 "subnet-def",
 "subnet-ghi",
 ],
 ),
 domain=metabase.CustomDomainArgs(
 hosted_zone_name="example.com",
 domain_name="metabase.example.com",
 ))

pulumi.export("url", metabase_service.dns_name)

package main

import (
 "github.com/pulumi/pulumi-metabase/sdk/go/metabase"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 metabaseService, err := metabase.NewMetabase(ctx, "metabaseService", &metabase.MetabaseArgs{
 VpcId: pulumi.String("vpc-123"),
 Networking: &metabase.NetworkingArgs{
 EcsSubnetIds: pulumi.StringArray{
 pulumi.String("subnet-123"),
 pulumi.String("subnet-456"),
 },
 DbSubnetIds: pulumi.StringArray{
 pulumi.String("subnet-789"),
 pulumi.String("subnet-abc"),
 },
 LbSubnetIds: pulumi.StringArray{
 pulumi.String("subnet-def"),
 pulumi.String("subnet-ghi"),
 },
 },
 Domain: &metabase.CustomDomainArgs{
 HostedZoneName: pulumi.String("example.com"),
 DomainName: pulumi.String("metabase.example.com"),
 },
 })
 if err != nil {
 return err
 }
 ctx.Export("url", metabaseService.DnsName)
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Metabase = Pulumi.Metabase;

return await Deployment.RunAsync(() =>
{
 var metabaseService = new Metabase.Metabase("metabaseService", new()
 {
 VpcId = "vpc-123",
 Networking = new Metabase.Inputs.NetworkingArgs
 {
 EcsSubnetIds = new[]
 {
 "subnet-123",
 "subnet-456",
 },
 DbSubnetIds = new[]
 {
 "subnet-789",
 "subnet-abc",
 },
 LbSubnetIds = new[]
 {
 "subnet-def",
 "subnet-ghi",
 },
 },
 Domain = new Metabase.Inputs.CustomDomainArgs
 {
 HostedZoneName = "example.com",
 DomainName = "metabase.example.com",
 },
 });

 return new Dictionary<string, object?>
 {
 ["url"] = metabaseService.DnsName,
 };
});

name: metabase-yaml
runtime: yaml
resources:
 metabaseService:
 type: "metabase:index:Metabase"
 properties:
 vpcId: "vpc-123"
 networking:
 ecsSubnetIds: [ "subnet-123", "subnet-456" ]
 dbSubnetIds: [ "subnet-789", "subnet-abc" ]
 lbSubnetIds: [ "subnet-def", "subnet-ghi" ]
 domain:
 hostedZoneName: "example.com"
 domainName: "metabase.example.com"
outputs:
 url: ${metabaseService.dnsName}

Delivery in 30 minutes or less

To get started unlocking your data with Metabase, head on over to the Metabase Package. Follow the quick walkthrough and have your Metabase service running in 30 minutes or less (in most cases). Once your service is up and running, we recommend popping over the Metabase Documentation to learn How to Set Up Metabase.

If you encounter an issue or have a feature request, please file an issue in the Package’s Github Repo.

Pulumi Interstellar

Zack Chase — Thu, 01 Apr 2021 00:00:00 +0000

Earth is just the beginning. We are putting down the foundations of space so our children can build their future. At Pulumi, we are committed to making life multi-planetary. We are excited to announce Pulumi Interstellar, a collection of resource providers that will help us reach the future of a space-faring and multi-planet species.

For our first offering of space related resource providers, we have aligned closely with the current priorities in the space industry today:

Getting to New Planets
Making Planets Habitable

We believe that solving those two areas for our customers will enable people to build the future they imagine, on the planets they desire to inhabit. One challenging aspect of achieving our goals is that each new planet represents a different operational environment.

One might think the right solution for provisioning and managing resources in space would be a proprietary domain-specific language designed from the ground up to operate in these new environments. However 10 out of 10 leading rocket scientists confirm that the heterogeneous diversity of existing programming languages allows us to move faster and with less frustration.

In the following sections we will take a closer look at the two main areas defined above and how you too can begin building the future in a familiar programming language like JavaScript, TypeScript, Python, Go, or .NET.

Rockets As Code

Rockets are the first step to making humans a multi-planet species, they enable you and your resources to travel to new planets. Today, we are extremely excited to introduce the Pulumi Rocket Provider, which allows users to configure rocket fuel requirements, initiation sequences, configure deployment payloads, reserve seats, and much more.

In the initial release of the Rocket Provider we are happy to enable two key use cases: Resource Deployment and Planetary Travel.

Resource Deployment

import * as tyrell from "@pulumi/tyrell"

const engineId = tyrell.rocket.getEngine({
 owners: ["420"],
 mostRecent: true,
 filters: [{name: "SuperDraco", values: ["tyrell/engines/super-draco-15.03-*"]}],
});

const spaceship = new tyrell.spaceship.Instance("Edward Isreal", {
 instanceType: tyrell.spaceship.InstanceTypes.interplanetary,
 engine: engineId,
 capability: "kuiper_belt",
}];

from pulumi_tyrell import rocket, spaceship

engine_id = rocket.getEngine(
 owners=["420"],
 most_recent=true,
 filters="""[
 {name: "SuperDraco", values: ["tyrell/engines/super-draco-15.03-*"]}
 ]""",
)

spaceship = spaceship.Instance("Edward Isreal",
 instanceType=spaceship.instance_types.interplanetary,
 engine=engineId,
 capability="kuiper_belt"
)

using Pulumi;
using Tyrell = Pulumi.Tyrell;

class MyStack : Stack
{
 public MyStack()
 {
 var engineId = new Tyrell.Rocket.GetEngine(new Tyrell.Rocket.GetEngineArgs
 {
 Owners =
 {
 "420",
 }
 MostRecent = true,
 Filters = {
 { "name": "SuperDraco", "values": ["tyrell/engines/super-draco-15.03-*"] },
 }
 });

 var spaceship = new Tyrell.Spaceship.Instance("", new Tyrell.Spaceship.InstanceArgs{
 InstanceType = Tyrell.Spaceship.InstanceTypes.Interplanetary,
 EngineId = engineId,
 Capability = "kuiper_belt",
 });
 }

}

package main

import (
 "github.com/pulumi/pulumi-tyrell/sdk/v3/go"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 engineId, err := tyrell.Rocket.GetEngine(&tyrell.Rocket.GetEngineArgs{
 Owners: []string{"420"},
 MostRecent: true,
 Filters: []RocketEngineFilter{
 RocketEngineFilter{
 Name: "SuperDraco",
 Values: []string{"tyrell/engines/super-draco-15.03-*"},
 },
 },
 })
 if err != nil {
 return err
 }

 spaceship, err := tyrell.Spaceship.Instance("Edward Isreal", &tyrell.Spaceship.InstanceArgs{
 InstanceType: tyrell.Spaceship.InstanceTypes.Interplanetary,
 EngineId: engineId,
 Capability: "kuiper_belt",
 })
 if err != nil {
 return err
 }

 return nil
 })
}

On Day 1 you can deploy resources via reusable rockets. By declaring a payload on your Rocket, you tell Pulumi to deliver the payload to the location specified. Resource Deployments will be particularly useful when configuring a new planet in the initial stages, as you can deploy new resources and terraforming equipment in a consistent, repeatable, and scalable manner.

Once the rocket arrives at its destination, Pulumi’s Space AI Assistant, nicknamed IRWIN, will take control to ensure your resources are deployed efficiently and correctly. By using Resource Deployments you can ensure your infrastructure and workloads are up and running before arriving on your new planet.

Planetary Travel

const config = pulumi.Config();
const passengers = config.require("passengers");

const seats = tyrell.tyrell.PassengerGroup("Robinsons", {
 spaceshipId: spaceship.then(it => it.id),
 seats: passengers,
 departure: "Luna",
 destination: "Mars",
 berth: "Colonist",
}];

config = pulumi.Config()
passengers = config.require("passengers")

seats = tyrell.tyrell.PassengerGroup("Robinsons",
 spaceshipId=spaceship.id,
 seats=passengers,
 departure="Luna",
 destination="Mars",
 berth="Colonist",
)

var config = Pulumi.Config();
var passengers = config.require("passengers");

var seats = new Tyrell.Tyrell.PassengerGroup("Robinsons", new Tyrell.Tyrell.PassengerGroupArgs
{
 SpaceshipId = spaceship.id,
 Seats = passengers,
 Departure = "Luna",
 Destination = "Mars",
 Berth = "Colonist",
});

config := pulumi.Config()
passengers := config.require("passengers")

seats, err := tyrell.PassengerGroup("Robinsons", &tyrell.PassengerGroupArgs{
 SpaceshipId: spaceship.id,
 Seats: passengers,
 Departure: "Luna",
 Destination: "Mars",
 Berth: "Colonist",
})

Once a planet is habitable and configured to your specifications, you will need a way to travel to your new home. Simply adding a seats argument to any Rocket will reserve those seats (as they eventually become available).

When your program completes deployment, you will receive a reservation confirmation via the email you used to register for the Pulumi Service. If you are an open source user, you will need to provide a valid email address via the --confirmation-email flag to receive your confirmation email.

Terraforming As Code

One of the most challenging obstacles preventing humans from becoming a multi-planet species is that Earth is the only planet in the known solar system that supports life. While there are many options for making other planets liveable, the team at Pulumi believes in the two step model towards habitability.

First, we need to provision dome-based communities in a repeatable and scalable manner. Once we have established our dome communities, we can then terraform any planet using the latest technology, i.e., nuclear missiles.

Dome-Based Communities

import * as weyland from "@pulumi/weyland-yutani"

const domeId = weyland.dome.getDome({
 owners: ["099720109477"],
 mostRecent: true,
 filters: [{ name: "name", values: ["weyland-yutanu/terraforming/acheron-*"] }],
}).then(it => it.id);

const terradome = weyland.dome,Instance("Hadley's Hope", {
 instanceType: weyland.dome.InstanceType.LV_246,
 dome: domeId,
 type: "atmospheric-processor",
}];

from pulumi_weyland_yutani import dome

dome = dome.get_dome(
 owners=["099720109477"],
 most_recent=true,
 filters="""[
 {name: "BasicDome", values: ["weyland-yutanu/terraforming/acheron-*"]}
 ]""",
)

terradome = dome.Instance("Hadley's Hope",
 instance_type=dome.instance_type.lv_246,
 dome=dome.id,
 type="atmospheric-processor",
)

using Pulumi;
using WeylandYutani = Pulumi.WeylandYutani;

class MyStack : Stack
{
 public MyStack()
 {
 var dome = new WeylandYutani.Dome.GetDome(new WeylandYutani.Dome.GetDomeArgs
 {
 Owners =
 {
 "099720109477",
 }
 MostRecent = true,
 Filters = {
 { "name": "BasicDome", "values": ["weyland-yutanu/terraforming/acheron-*"] },
 }
 });

 var terradome = new WeylandYutani.Dome.Instance("Hadley's Hope", new Weyland.Yutani.Dome.InstanceArgs
 {
 InstanceType = Weyland.Dome.InstanceType.LV_246,
 Dome = dome.Id,
 Type = "atmospheric-processor",
 });
 }

}

package main

import (
 "github.com/pulumi/pulumi-weyland-yutani/sdk/v3/go"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 dome, err := weyland.Dome.GetDome(&weyland.Dome.GetDomeArgs{
 Owners: []string{"099720109477"},
 MostRecent: true,
 Filters: []RocketEngineFilter{
 RocketEngineFilter{
 Name: "BasicDome",
 Values: []string{"weyland-yutanu/terraforming/acheron-*"},
 },
 },
 })
 if err != nil {
 return err
 }

 terradome, err := weyland.Dome.Instance("", &weyland.Dome.InstanceArgs{
 InstanceType: weyland.Dome.InstanceType.LV_246,
 Dome: dome.Id,
 Type: "atmospheric-processor",
 })
 if err != nil {
 return err
 }

 return nil
 })
}

The first step to establish humanity on other planets is to provision dome-based communities and build critical infrastructure and resources while we work to terraform the planet. Out of the box, Pulumi provides boilerplate communities that are flexible and resilient enough to support any use case.

Nuclear Terraforming

While we will not be able to support nuclear terraforming on Day 1, we are working extremely diligently to ensure our customers have access to state of the art terraforming techniques. We hope to bring a preview of Nuclear Terraforming to customers in early Q3 of 2022. Be on the lookout for monthly updates as we develop this new groundbreaking technology.

How To Get Started

Getting started is as simple as getting started with any other Pulumi resource provider, you just need to download the package with your preferred language’s package manager, configure the provider, and then start building. All space resources are billed via Dogecoin, so you will need to provide a wallet address in your provider’s arguments.

$ pulumi config set --secret doge DilB3JrTyGm55FiMyRlhM7NZmfj0J9QVX

Don’t worry, we’ll encrypt your wallet address. Your Dogecoin wallet will go straight to your provider and Pulumi never receives or handles your secrets.

Summary

Today is one of the most exciting days in Pulumi’s short history and we are incredibly excited to partner with our customers and leading space companies to make our multi-planetary dreams a reality. We cannot wait to see what you’ll build!

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}" >