Pulumi Blog: Fargate

Deploying an OAuth Server for Netlify's CMS

Zephyr Zhou — Wed, 09 Sep 2020 00:00:00 +0000

Some of the code in this post is out of date. See the AWS guides for an updated overview and examples.

In our previous post, we deployed our CMS app on AWS instead of Netlify. We couldn’t use Netlify’s Identity Service, which manages GitHub access to Netlify CMS, because we deployed on AWS. As a result, we needed to implement an external OAuth Server.

We used Netlify’s Go example to deploy on ECS Fargate and configure the domain and certificate. To deploy the application on Fargate, we used a Typescript Pulumi project. This is a polyglot application where the OAuth server is implemented in Go and the infrastructure is deployed with Typescript. We’ll show how we accomplished the deployment.

Backend is a package that supports communications between Netlify CMS and repositories like GitHub, GitLab, and Bitbucket. The Pulumi example code uses backend to authenticate to the CMS. The OAuth Server also enables authorization for GitLab and Bitbucket by changing the callback URL to https://{{YOUR_OAUTH_SERVER_URL}}/callback/{{YOUR_BACKEND_NAME}}. To learn more about configuring OAuth Server, refer to the Environment Variable and Pulumi Stack Configuration section.

Building the OAuth Server

Netlify’s CMS website provides External OAuth Client examples for various languages and platforms. We used the Go example as a template for our server.

In Netlify’s example, the ./dotenv/dotenv.go retrieves environment variables from a file. The main.go file uses goth to instantiate the OAuth provider. The ./randstr/randstr.go file generates a random string for the SESSION_SECRET environment variable, which is used for authentication. However, Pulumi can implement both functions, replacing the code in goth by using Pulumi’s random package.

// Create a random string and also mark its `result` property as a secret,
// so it is not stored in plaintext in the stack's state.
const sessionSecretRandomString = new random.RandomPassword("random", {
 length: 32,
}, { additionalSecretOutputs: ["result"] });

Because we replaced those functions, we can remove the randstr and dotenv folders in the OAuth client example. Later, we’ll show how to pass the environment variables when we create the Fargate service.

Setting GitHub Scope

The default GitHub scope in Netlify’s example is read-only. When we call the github.New function, read and write access must be set to edit a target repository.

We set GITHUB_SCOPE to public_repo, granting read and write access to a public repository. The GITHUB_SCOPE value is read and parsed when the Fargate Service is created, which is set in the Github.New function in the OAuth server. We can also set GITHUB_SCOPE to a list of scope values.

githubScope := os.Getenv("GITHUB_SCOPE")
 if githubScope == "" {
 goth.UseProviders(
 github.New(
 os.Getenv("GITHUB_KEY"), os.Getenv("GITHUB_SECRET"),
 // concatenate with the host name
 fmt.Sprintf("%s/callback/github", host),
 "public_repo",
 ),
 bitbucket.New(
 os.Getenv("BITBUCKET_KEY"), os.Getenv("BITBUCKET_SECRET"),
 fmt.Sprintf("%s/callback//bitbucket", host),
 ),
 gitlabProvider,
 )
 } else {
 scopeArray := strings.Split(githubScope, ",")
 goth.UseProviders(
 github.New(
 os.Getenv("GITHUB_KEY"), os.Getenv("GITHUB_SECRET"),
 // concatenate with the host name
 fmt.Sprintf("%s/callback/github", host),
 scopeArray...,
 ),
 bitbucket.New(
 os.Getenv("BITBUCKET_KEY"), os.Getenv("BITBUCKET_SECRET"),
 fmt.Sprintf("%s/callback//bitbucket", host),
 ),
 gitlabProvider,
 )
 }

Creating the OAuth Server Image

To build our OAuth server container, replace the working directory in the Dockerfile with the path to your repository. Setting the working directory copies the application when Docker builds the image.

WORKDIR /go/src/github.com/pulumi/aws-ts-netlify-cms-and-oauth/cms-oauth

Infrastructure

Deploying to Fargate

We’ll use the Hello Fargate Example as a template to deploy the OAuth server to AWS Fargate. This example, written in Typescript, creates an ECS cluster, an Application Load Balancer (alb) with a listener, and a Fargate Service. We’ll change the script to deploy our OAuth server Docker image.

We set the alb port to 443 to use HTTPS, which is more secure when passing the CMS access token.

// Define an ec2 application load balancer alb to distribute incomming application traffic across multiple targets, such as EC2 instances, in multiple Availability Zones.
const alb = new awsx.elasticloadbalancingv2.ApplicationLoadBalancer(
 "net-lb", { external: true, securityGroups: cluster.securityGroups });

// alb need a listener to listen to 443 the standard port for the HTTPS traffic, certificate is using the certificate we created above
const web = alb.createListener("web", {
 port: 443,
 external: true,
 protocol: "HTTPS",
 certificateArn: certificate.arn
});

We create a target group for the load balancers to distribute traffic. The target group port defaults port 80 if it’s not set.

let inputTargetGroupPort: pulumi.Input<number> = cmsStackConfig.targetGroupPort!;

if (inputTargetGroupPort === undefined) {
 inputTargetGroupPort = 80;
}

// when Listener Rule is satisfied then traffic is route to this target group.
const tg = alb.createTargetGroup("oauth-tg", {
 port: inputTargetGroupPort,
 loadBalancer: alb
});

We can also set the target group port as an environment variable with pulumi config set cms-OAuth:targetGroupPort. The load balancer implements a listener and a rule to forward all requests to the target group.

// when the request are forwarded then every requests are send to the target group we created
new awsx.lb.ListenerRule("oauth-listener-rule", web, {
 actions: [{
 type: "forward",
 targetGroupArn: tg.targetGroup.arn,
 }],
 conditions: [{
 field: "path-pattern",
 values: "/*", //wildcard says every request would be send
 }],
});

Environment Variables and Pulumi Stack Configuration

The Fargate service requires several environment variables to start. Here are the variables we need to configure:

HOST is the URL for the OAuth Server, and we set it by running pulumi config set pulumi-website-cms:targetDomain <OAuth server URL>
SESSION_SECRET is generated by using Pulumi’s random package
GITHUB_SECRET and GITHUB_TOKEN are provided by GitHub when the GitHub OAuth application is registered. Netlify provides instructions for obtaining these credentials
The targetDomain is the URL of our CMS
GITHUB_SCOPE sets the type of access the CMS has for the target repository

We can set the github key and secret in the stack configuration. The –secret flag encrypts your GitHub secret. It is good practice to use the –secret flag, because appending the secret in the command line stores it in the command history. The –secret flag lets you enter the secret at the prompt, which hides the value.

$ pulumi config set netlify-cms-oauth-provider-infrastructure:githubKey {{YOUR_GITHUB_KEY}}
$ pulumi config set --secret netlify-cms-oauth-provider-infrastructure:githubSecret
$ {{YOUR_GITHUB_SECRET}}

When the Fargate service is created, it uses the environment variables set in the Pulumi project file and generates the session secret.

// Create a Fargate service task that can scale out.
const appService = new awsx.ecs.FargateService("app-svc", {
 cluster,
 taskDefinitionArgs: {
 container: {
 image: img,
 memory: 128 /*MB*/,
 portMappings: [ tg ],
 environment: [
 {
 name: "HOST",
 // The target domain which would concatenate with callbacks in main.go
 value: pulumi.interpolate `https://${cmsStackConfig.targetDomain}`
 },
 {
 name: "SESSION_SECRET",
 value: sessionSecretRandomString.result
 },
 {
 name: "GITHUB_KEY",
 value: cmsStackConfig.githubKey
 },
 {
 name: "GITHUB_SECRET",
 value: cmsStackConfig.githubSecret
 },
 {
 name: "TARGET_PORT",
 value: pulumi.interpolate `${inputTargetGroupPort}`
 },
 {
 name: "GITHUB_SCOPE",
 value: inputGithubScope
 },
 ]
 },
 },
 desiredCount: 1,
});

The environment variables are passed to the OAuth server in main.go. For example, the callback URL is the targetDomain with callback and the repository provider appended, e.g., https://<targetDomain>/callback/github/. You can also specify a different repository provider, such as Bitbucket or GitLab.

 github.New(
 os.Getenv("GITHUB_KEY"), os.Getenv("GITHUB_SECRET"),
 // concatenate with the host name
 fmt.Sprintf("%s/callback/github", host),
 "public_repo",
 )

We also pass TARGET_PORT to targetGroupPort, which is the port used to serve the application. We can check if this variable is set in main.go if we need to test locally without Pulumi.

targetGroupPort := os.Getenv("TARGET_PORT")
 if targetGroupPort == "" {
 targetGroupPort = "80"
 }
 listenPort := ":" + targetGroupPort
 fmt.Print("Started running on", listenPort, "\n")
 // listen on port 80 where we created the target group
 fmt.Println(http.ListenAndServe(listenPort, nil))

Create a Certificate for the OAuth Server

We use the same process for creating a certificate that’s documented in our previous post, but we use only the code for creating the certificate and discard the rest. The certificate used with the load balancer must be set to the same region as the fully qualified domain name (FQDN). The domain name in the certificate must be validated.

// Get a east provider
const eastRegion = new aws.Provider("east", {
 profile: aws.config.profile,
 region: "us-east-1", // Per AWS, ACM certificate must be in the us-east-1 region.
});

// Creating a Certificate for the given domain.
const certificate = new aws.acm.Certificate("certificate", {
 domainName: cmsStackConfig.targetDomain,
 validationMethod: "DNS",
}, { provider: eastRegion });

// Split given domain in the configuration to domain and subdomain
const domainParts = getDomainAndSubdomain(cmsStackConfig.targetDomain);

// Get the zone of the given domain
const hostedZoneId = aws.route53.getZone({ name: domainParts.parentDomain }, { async: true }).then(zone => zone.zoneId);

// The temporation record for the validation domain has 10 to be live
const tenMinutes = 60 * 10;

/**
 * Create a DNS record to prove that we _own_ the domain we're requesting a certificate for.
 * See https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-validate-dns.html for more info.
 */
const certificateValidationDomain = new aws.route53.Record(`${cmsStackConfig.targetDomain}-validation`, {
 name: certificate.domainValidationOptions[0].resourceRecordName,
 zoneId: hostedZoneId,
 type: certificate.domainValidationOptions[0].resourceRecordType,
 records: [certificate.domainValidationOptions[0].resourceRecordValue],
 ttl: tenMinutes,
});

We configure the Alias Record with the application load balancer settings.

// Creates a new Route53 DNS record pointing the domain to the CloudFront distribution.
function createAliasRecord(
 targetDomain: string, lb: awsx.elasticloadbalancingv2.ApplicationLoadBalancer): aws.route53.Record {
 const domainParts = getDomainAndSubdomain(targetDomain);
 const hostedZoneId = aws.route53.getZone({ name: domainParts.parentDomain }, { async: true }).then(zone => zone.zoneId);
 return new aws.route53.Record(
 targetDomain,
 {
 name: domainParts.subdomain,
 zoneId: hostedZoneId,
 type: "A",
 aliases: [
 {
 name: lb.loadBalancer.dnsName,
 zoneId: lb.loadBalancer.zoneId,
 evaluateTargetHealth: true,
 },
 ],
 });
}
// Create the aliasRecord with targetdomain and application load balancer
const aRecord = createAliasRecord(cmsStackConfig.targetDomain, alb);

GitHub Workflow (Optional)

We can automate our build with GitHub Workflow. The build-and-deploy.yml file in cms-oauth/.github/workflows/ is used by GitHub Actions to build our application. Note that the workflow uses the repository secret set in GitHub.

Finishing Up

Our last task is to update the CMS configuration file. We combine the CMS and CMS OAuth Server deployment by setting site_domain and URL in ./public/config.yml of the CMS folder.

backend:
 name: github
 # Replace this with the GitHub repo you want to make change
 repo: github-username/target-github-repo
 # This site_domain and base_url are placeholders, replace with the correct one.
 site_domain: https://some-cms-domain.com # CMS's domain
 base_url: https://some-oauth-domain.com # OAuth Server's domain

The site_domain is the CMS URL, and the base_url is the OAuth Server URL. They are the same as the targetDomain variable we set in the stack configuration.

Up and Running

Congratulations on building and deploying both CMS web application and the OAuth client-server! If everything works perfectly, you should see the Login with GitHub button and click it to redirect to the GitHub login page.

People with access can now use the CMS.

You can find the project code for the CMS and the OAuth Client Server on GitHub.

Special Thanks to:

Tony Alves @talves for providing the template to separate CMS as a stand-alone React App
Igor Kuznetsov @igk1972 for providing go code example for providing the OAuth Client-Server source code
Paul Stack @stack72 for developing Pulumi example of deploying Dockerized App using ECS Fargate and Pulumi example of deploying the static website on AWS
Everyone at Pulumi who helped me out with this project.

ECS vs Fargate vs EKS: The Lowdown on Containers in AWS

Joe Duffy — Thu, 20 Jun 2019 00:00:00 +0000

Some of the code in this post is out of date. See the AWS guides for an updated overview and examples.

An updated breakdown of when to prefer Fargate and when EC2 can be found in Fargate vs EC2

Amazon offers multiple solutions for running containers in AWS, through its managed Elastic Container Service (ECS). This includes three major approaches: ECS managed automatically with Fargate, ECS backed by EC2 instances, and Elastic Kubernetes Service (EKS), delivering the full power of Kubernetes. It's not always easy to choose between these, so in this article we provide some basic guidance on the tradeoffs you'll encounter when choosing.

The Options: ECS vs Fargate vs EKS

At Pulumi, we work with customers to deploy AWS applications all the time, from early development to scaling production environments worldwide, using infrastructure as code and often continuous delivery. During this, we’ve run across a breadth of container workloads and requirements, and developed best practices, which this article distills into some basic advice. As with any advice, the right answer depends on your scenario’s specific requirements.

Here are the three major options, roughly speaking from easiest to hardest:

Option 1 - ECS Fargate

Amazon ECS with Fargate launch type allows you to deploy and scale your containerized workloads, automatically, without needing to manage clusters of EC2 instances by hand. You focus on your Dockerized applications, with minimal extra declarative metadata about the requirements of those applications, and not the details of the underlying compute itself. One could say this is “serverless containers.” Of course, there are servers running your containers, however the ECS control plane makes informed decisions on your behalf about when to add capacity, where, and by how much.

To run a workload in ECS, you use a standard Dockerfile to author, build, and publish your container image, and then run that image inside your ECS cluster. Tasks and services are authored using ECS definitions, which specify the container image to run, instance count, and any CPU, memory, networking, and disk requirements your workload has.

Using Pulumi, these definitions can be easily expressed declaratively in code, like so:

import * as awsx from "@pulumi/awsx";

// Create a load balancer on port 80 and spin up two instances of Nginx.
const lb = new awsx.lb.ApplicationListener("nginx", { port: 80 });
const nginx = new awsx.ecs.FargateService("nginx", {
 taskDefinitionArgs: {
 containers: {
 nginx: {
 image: "nginx",
 memory: 128,
 portMappings: [ lb ],
 },
 },
 },
 desiredCount: 2,
});

// Export the load balancer's address so that it's easy to access.
export const url = lb.endpoint.hostname;

This example runs 2 load balanced instances of an NGINX web server using ECS Fargate.

ECS Fargate is often a great choice when getting started, when you don’t have stringent requirements around cost, and/or if you’re a small to medium sized organization who prefers simpler solutions. Many teams are successful running it scale too, however Fargate doesn’t give you as much control over persistent storage, privileged containers, custom scheduling, or special VM types like GPUs or FPGAs. For those, EC2 backed ECS is a better choice.

Option 2 - ECS Backed by EC2 Instances

Amazon ECS with EC2 launch type gives you similar capabilities to ECS managed by Fargate – in that you can deploy containerized workloads, declaratively specify their requirements, and let ECS do the placement and overall scheduling of them – but with full control over the compute environment those containers run in. That includes control over the precise numbers of servers, AMIs they are running, and their auto-scaling policies.

To use an EC2 backed cluster, you will need to create a cluster and configure the EC2 launch and autoscaling configurations, and then simply use the awsx.ecs.EC2Service class to create your service instead. Other than these differences, the resulting code would look very similar to Option 1 - ECS Fargate above.

For sophisticated AWS organizations who already know how to fine tune and run compute at scale efficiently and cost effectively, managing your ECS cluster’s EC2 instances explicitly is often a good choice. The good news is that you can easily start with Fargate, and then over time, shift to managing the EC2 compute by hand if you prefer. Managing your cluster amounts to managing fleets of EC2 instances, CloudWatch logging, and standard AWS services.

Option 3 - Elastic Kubernetes Service (EKS)

Amazon Elastic Kubernetes Service (EKS) offers a very different approach to running containerized workloads than ECS Fargate or EC2. EKS is a fully managed offering that runs Kubernetes inside of your AWS account, making it easier to operate Kubernetes, in addition to integrating with many AWS services like ELB for load balancing, IAM for cluster authentication, and CloudWatch for logging and diagnostics. Using EKS, you don’t need to explicitly provision, manage, and upgrade the Kubernetes control plane’s configuration, compute, and storage.

Kubernetes is an open standard for running containers across a variety of public and private cloud environments. Most public cloud vendors now offer an EKS-like managed offering, including Azure (AKS), Google Cloud (GKE), and Digital Ocean, and the availability of on-premises private cloud configurations enables hybrid cloud for large organizations. Kubernetes in inherently more complicated to operate than the equivalent ECS solutions mentioned earlier, because it requires becoming an expert in Kubernetes, which requires more specialization in its tools and management practices. But the potential payoff is larger for organizations wanting to go “all in” on cloud native technologies to facilitate either multi-cloud or hybrid computing – challenges most large enterprises are facing today.

In the EKS case, as with ECS, we continue using standard Dockerfiles, however we use the Kubernetes object model to configure and run workloads. Instead of task definitions, you use Kubernetes objects like Namespaces, Pods, Deployments, and Services. This full API is available to specify in code using Pulumi, like so:

import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";

// Create an EKS cluster with the default configuration.
const cluster = new eks.Cluster("my-cluster");

// Deploy a small canary service (NGINX), to test that the cluster is working.
const appName = "my-app";
const appLabels = { appClass: appName };
const deployment = new k8s.apps.v1.Deployment(`${appName}-dep`, {
 metadata: { labels: appLabels },
 spec: {
 replicas: 2,
 selector: { matchLabels: appLabels },
 template: {
 metadata: { labels: appLabels },
 spec: {
 containers: [{
 name: appName,
 image: "nginx",
 ports: [{ name: "http", containerPort: 80 }]
 }],
 }
 }
 },
}, { provider: cluster.provider });
const service = new k8s.core.v1.Service(`${appName}-svc`, {
 metadata: { labels: appLabels },
 spec: {
 type: "LoadBalancer",
 ports: [{ port: 80, targetPort: "http" }],
 selector: appLabels,
 },
}, { provider: cluster.provider });

// Export the URL for the load balanced service.
export const url = service.status.loadBalancer.ingress[0].hostname;

// Export the cluster's kubeconfig.
export const kubeconfig = cluster.kubeconfig;

Like the ECS example earlier, this spins up 2 load balanced instances of the NGINX web server.

You will find that the entirety of Kubernetes is more feature rich and offers more control over scheduling and scaling your workloads, but with that richness also comes added complexity that will increase training, and time to ramp up, in addition to ongoing maintenance costs. To learn more about Pulumi’s support for Kubernetes, go here.

Private ECR Container Registries

In all of these cases, Amazon Elastic Container Registry (ECR) lets you publish Docker container images to a privately hosted repository inside your AWS account. The benefit of this is that images are stored within your AWS account, where access is governed by IAM and, where they are stored closer to your workloads (maximizing push/pull efficiency). This is an alternative approach to, say, publishing images to the Docker Hub which entails a different authentication and pushing and pulling images over the Internet, where performance can be an issue.

Next Steps

Pulumi Crosswalk for AWS supports three main options for running containers in AWS – ECS Fargate, ECS with EC2 instances, and EKS – each of which integrates deeply with other AWS services like IAM, ELB, and CloudWatch. It’s possible to use any of these services without Pulumi, but there are many benefits to Pulumi’s infrastructure as code.

In short, to get your containers up on AWS, in an easy yet production-ready way, get started for free with Pulumi for AWS today!

After getting started, here are some additional resources to go deeper on specific topics: