Pulumi Blog: Nishi Davidson

Persisting Kubernetes workloads with Amazon EFS CSI volumes

Nishi Davidson — Mon, 15 Jul 2019 00:00:00 +0000

Some parts of this blog post are out-of-date. As an alternative, please refer to the EFS CSI Helm Chart and Pulumi’s support for deploying helm charts

The Amazon Elastic File System Container Storage Interface (CSI) Driver implements the CSI specification for container orchestrators to manage the lifecycle of Amazon EFS filesystems. The CSI specification defines an interface along with the minimum operational and packaging recommendations for a storage provider to implement a CSI compatible plugin. The interface declares the RPCs that a plugin must expose. The CSI drivers are the right mechanism to work with, when using a cloud storage component with Kubernetes workloads. Amazon Elastic File System (Amazon EFS) provides parallel shared access through a standard file system interface to Amazon EC2 instances and Linux-based workloads without disrupting your applications. EFS is a regional service storing data across multiple Availability Zones (AZs) for high availability and durability.

The AWS EFS CSI Driver is a Kubernetes SIG-AWS subproject. The AWS EFS CSI Driver triggers the creation of out-of-tree CSI volume plugins in any Kubernetes cluster. Thereafter Kubernetes storage classes, persistent volumes and claims can be used by a Kubernetes workload to access the EFS CSI volume plugin as before.

In this blog, we will work through an example that shows how to use AWS EFS CSI storage components with Kubernetes workloads running on Amazon EKS worker nodes using Pulumi libraries (EKS, AWS, and AWSX). These example steps are running on OS X.

Step 1: Initialize Pulumi Project and Stack for your organization:

Install Pulumi CLI and set up your AWS credentials. Initialize a new Pulumi project from available templates. We use aws-typescript template here to install all library dependencies.

We will work with two Pulumi stacks in this example, one for the Amazon EKS cluster and AWS EFS CSI components called k8sinfra. The other for the application and its storage class, persistent volume and persistent volume claim called app. The AWS EFS CSI (Container Storage Interface) is based on the initial AWS EFS CSI Driver work done by Kubernetes SIG-AWS.

$ brew install pulumi/tap/pulumi # download Pulumi CLI

$ mkdir k8sinfra && cd k8sinfra

$ pulumi new aws-typescript

$ npm install --save @pulumi/kubernetes @pulumi/eks

$ ls -la
drwxr-xr-x 10 nishidavidson staff 320 Jun 18 18:22 .
drwxr-xr-x+ 102 nishidavidson staff 3264 Jun 18 18:13 ..
-rw------- 1 nishidavidson staff 21 Jun 18 18:22 .gitignore
-rw-r--r-- 1 nishidavidson staff 32 Jun 18 18:22 Pulumi.dev.yaml
-rw------- 1 nishidavidson staff 91 Jun 18 18:22 Pulumi.yaml
-rw------- 1 nishidavidson staff 273 Jun 18 18:22 index.ts
drwxr-xr-x 95 nishidavidson staff 3040 Jun 18 18:22 node_modules
-rw-r--r-- 1 nishidavidson staff 50650 Jun 18 18:22 package-lock.json
-rw------- 1 nishidavidson staff 228 Jun 18 18:22 package.json
-rw------- 1 nishidavidson staff 522 Jun 18 18:22 tsconfig.json

Step 2: Create the EKS cluster, EFS endpoint and MountTargets:

The code below first declares an EKS cluster attached to two public subnets in a new VPC. We then declare EFS endpoint and mount the same to both subnets so that the EKS worker nodes can access the filesystem. This code should be pasted into index.ts.

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";

//* STEP 1: Create an EKS cluster, an EFS endpoint and mount the EFS endpoint in each public subnet of the EKS cluster

// Create an EKS Cluster
const vpc = new awsx.ec2.Vpc("vpc", {});
export const cluster = new eks.Cluster("eks-cluster", {
 vpcId: vpc.id,
 subnetIds: vpc.publicSubnetIds,
 instanceType: "t2.medium",
 version: "1.12",
 nodeRootVolumeSize: 200,
 desiredCapacity: 2,
 maxSize: 3,
 minSize: 2,
 deployDashboard: true,
 vpcCniOptions: {
 warmIpTarget: 4
 }
});

const nodeSecurityGroup = cluster.nodeSecurityGroup;
export const nodesubnetIds = cluster.core.subnetIds;
export const clusterName = cluster.eksCluster.name;
export const kubeconfig = cluster.kubeconfig.apply(JSON.stringify);

// Create an EFS endpoint
export const efsFilesystem = new aws.efs.FileSystem("MyEFS", {
 tags: { Name: "myEFS" }
});

export const efsFilesystemId = efsFilesystem.id;

// Create a mounttarget in each of the EKS cluster VPC's AZs so that EC2 instances across the VPC can access the filesystem
new aws.efs.MountTarget("MyEFS-MountTarget1", {
 fileSystemId: efsFilesystemId,
 securityGroups: [nodeSecurityGroup.id],
 subnetId: nodesubnetIds[0]
});

new aws.efs.MountTarget("MyEFS-MountTarget2", {
 fileSystemId: efsFilesystemId,
 securityGroups: [nodeSecurityGroup.id],
 subnetId: nodesubnetIds[1]
});

Step 3: Create the CSI Driver Node and Controller Components

Currently, only static provisioning for AWS EFS CSI drivers is supported by SIG-AWS, which implies the Amazon EFS filesystem needs to be manually created first. After that it can be mounted inside a container as a volume using the driver. The code below will allow you to deploy allow the CSI Driver components on the Amazon EKS cluster.

//* STEP 3: Install EFS CSI Driver Node and Controller components

// Install EFS CSI Driver node components
const svcacntcsiNode = new k8s.core.v1.ServiceAccount(
 "csi-node-sa",
 {
 metadata: { name: "csi-node-sa", namespace: "kube-system" }
 },
 { provider: cluster.provider }
);

const clusterolecsiNode = new k8s.rbac.v1.ClusterRole(
 "csi-node",
 {
 metadata: { name: "csi-node", namespace: "default" },
 rules: [
 {
 apiGroups: [""],
 resources: ["secrets"],
 verbs: ["get", "list"]
 },
 {
 apiGroups: [""],
 resources: ["nodes"],
 verbs: ["get", "list", "update"]
 },
 {
 apiGroups: [""],
 resources: ["namespaces"],
 verbs: ["get", "list"]
 },
 {
 apiGroups: [""],
 resources: ["persistentvolumes"],
 verbs: ["get", "list", "watch", "update"]
 },
 {
 apiGroups: ["storage.k8s.io"],
 resources: ["volumeattachments"],
 verbs: ["get", "list", "watch", "update"]
 },
 {
 apiGroups: ["csi.storage.k8s.io"],
 resources: ["csinodeinfos"],
 verbs: ["get", "list", "watch", "update"]
 }
 ]
 },
 { provider: cluster.provider }
);

const clusterolebindingcsiNode = new k8s.rbac.v1.ClusterRoleBinding(
 "csi-node",
 {
 metadata: { name: "csi-node", namespace: "default" },
 subjects: [
 {
 kind: "ServiceAccount",
 name: "csi-node-sa",
 namespace: "kube-system"
 }
 ],
 roleRef: {
 kind: "ClusterRole",
 name: "csi-node",
 apiGroup: "rbac.authorization.k8s.io"
 }
 },
 { provider: cluster.provider }
);

const daemonsetcsiNode = new k8s.apps.v1beta2.DaemonSet(
 "efs-csi-node",
 {
 metadata: { name: "efs-csi-node", namespace: "kube-system" },
 spec: {
 selector: { matchLabels: { app: "efs-csi-node" } },
 template: {
 metadata: { labels: { app: "efs-csi-node" } },
 spec: {
 serviceAccount: "csi-node-sa",
 hostNetwork: true,
 containers: [
 {
 name: "efs-plugin",
 securityContext: {
 privileged: true
 },
 image: "amazon/aws-efs-csi-driver:latest",
 imagePullPolicy: "Always",
 args: ["--endpoint=$(CSI_ENDPOINT)", "--logtostderr", "--v=5"],
 env: [{ name: "CSI_ENDPOINT", value: "unix:/csi/csi.sock" }],
 volumeMounts: [
 {
 name: "kubelet-dir",
 mountPath: "/var/lib/kubelet",
 mountPropagation: "Bidirectional"
 },
 { name: "plugin-dir", mountPath: "/csi" },
 { name: "device-dir", mountPath: "/dev" }
 ]
 },
 {
 name: "csi-driver-registrar",
 image: "quay.io/k8scsi/driver-registrar:v0.4.2",
 imagePullPolicy: "Always",
 args: [
 "--csi-address=$(ADDRESS)",
 "--mode=node-register",
 "--driver-requires-attachment=true",
 "--pod-info-mount-version=v1",
 "--kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)",
 "--v=5"
 ],
 env: [
 { name: "ADDRESS", value: "/csi/csi.sock" },
 {
 name: "DRIVER_REG_SOCK_PATH",
 value: "/var/lib/kubelet/plugins/efs.csi.aws.com/csi.sock"
 },
 {
 name: "KUBE_NODE_NAME",
 valueFrom: { fieldRef: { fieldPath: "spec.nodeName" } }
 }
 ],
 volumeMounts: [
 { name: "plugin-dir", mountPath: "/csi" },
 { name: "registration-dir", mountPath: "/registration" }
 ]
 }
 ],
 volumes: [
 {
 name: "kubelet-dir",
 hostPath: { path: "/var/lib/kubelet", type: "Directory" }
 },
 {
 name: "plugin-dir",
 hostPath: {
 path: "/var/lib/kubelet/plugins/efs.csi.aws.com/",
 type: "DirectoryOrCreate"
 }
 },
 {
 name: "registration-dir",
 hostPath: { path: "/var/lib/kubelet/plugins/", type: "Directory" }
 },
 {
 name: "device-dir",
 hostPath: { path: "/dev", type: "Directory" }
 }
 ]
 }
 }
 }
 },
 { provider: cluster.provider }
);

// Install EFS CSI Driver Controller components
const svcacctcsiController = new k8s.core.v1.ServiceAccount(
 "csi-controller-sa",
 {
 metadata: { name: "csi-controller-sa", namespace: "kube-system" }
 },
 { provider: cluster.provider }
);

const clusterrolecsiController = new k8s.rbac.v1.ClusterRole(
 "external-attacher-role",
 {
 metadata: { name: "external-attacher-role", namespace: "default" },
 rules: [
 {
 apiGroups: [""],
 resources: ["persistentvolumes"],
 verbs: ["get", "list", "watch", "update"]
 },
 {
 apiGroups: [""],
 resources: ["nodes"],
 verbs: ["get", "list", "watch"]
 },
 {
 apiGroups: ["storage.k8s.io"],
 resources: ["volumeattachments"],
 verbs: ["get", "list", "watch", "update"]
 }
 ]
 },
 { provider: cluster.provider }
);

const clusterrolebindingcsiController = new k8s.rbac.v1.ClusterRoleBinding(
 "csi-attacher-role",
 {
 metadata: { name: "csi-attacher-role", namespace: "default" },
 subjects: [
 {
 kind: "ServiceAccount",
 name: "csi-controller-sa",
 namespace: "kube-system"
 }
 ],
 roleRef: {
 kind: "ClusterRole",
 name: "external-attacher-role",
 apiGroup: "rbac.authorization.k8s.io"
 }
 },
 { provider: cluster.provider }
);

const statefulsetcsiController = new k8s.apps.v1beta1.StatefulSet(
 "efs-csi-controller",
 {
 metadata: { name: "efs-csi-controller", namespace: "kube-system" },
 spec: {
 serviceName: "efs-csi-controller",
 replicas: 1,
 template: {
 metadata: { labels: { app: "efs-csi-controller" } },
 spec: {
 serviceAccount: "csi-controller-sa",
 priorityClassName: "system-cluster-critical",
 tolerations: [{ key: "CriticalAddonsOnly", operator: "Exists" }],
 containers: [
 {
 name: "efs-plugin",
 image: "amazon/aws-efs-csi-driver:latest",
 imagePullPolicy: "Always",
 args: ["--endpoint=$(CSI_ENDPOINT)", "--logtostderr", "--v=5"],
 env: [
 {
 name: "CSI_ENDPOINT",
 value: "unix:///var/lib/csi/sockets/pluginproxy/csi.sock"
 }
 ],
 volumeMounts: [
 {
 name: "socket-dir",
 mountPath: "/var/lib/csi/sockets/pluginproxy/"
 }
 ]
 },
 {
 name: "csi-attacher",
 image: "quay.io/k8scsi/csi-attacher:v0.4.2",
 imagePullPolicy: "Always",
 args: ["--csi-address=$(ADDRESS)", "--v=5"],
 env: [
 {
 name: "ADDRESS",
 value: "/var/lib/csi/sockets/pluginproxy/csi.sock"
 }
 ],
 volumeMounts: [
 {
 name: "socket-dir",
 mountPath: "/var/lib/csi/sockets/pluginproxy/"
 }
 ]
 }
 ],
 volumes: [{ name: "socket-dir", emptyDir: {} }]
 }
 }
 }
 },
 { provider: cluster.provider }
);

Step 4: Deploy a sample app with the EFS volume mounts:

Once the step above is complete, you will be ready to deploy your k8s sample application, storage class, persistent volume and persistent volume claim. Lets create a new stack for this called k8sapp as follows:

$ mkdir k8sapp && cd k8sapp

$ pulumi new typescript

$ npm install --save @pulumi/eks @pulumi/kubernetes

Let’s now declare the storage class, pv, pvc and application pod to complete our k8s application set-up for this example. We will update index.ts file again and run pulumi up:

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

const env = pulumi.getStack();
const cluster = new pulumi.StackReference(`d-nishi/k8sinfra/${env}`);
const kubeconfig = cluster.getOutput("kubeconfig");
const efsFilesystemId = cluster.getOutput("efsFilesystemId");

const k8sProvider = new k8s.Provider("cluster", {
 kubeconfig: kubeconfig
});

//* STEP 5: Create a storage class, persistent volume and persistent volume claim

export const storageclassEFS = new k8s.storage.v1.StorageClass(
 "efs-sc",
 {
 metadata: { name: "efs-sc" },
 provisioner: "efs.csi.aws.com"
 },
 { provider: k8sProvider }
);

const pvEFS = new k8s.core.v1.PersistentVolume(
 "efs-pv",
 {
 metadata: { name: "efs-pv" },
 spec: {
 capacity: { storage: "5Gi" },
 volumeMode: "Filesystem",
 accessModes: ["ReadWriteOnce"],
 persistentVolumeReclaimPolicy: "Recycle",
 storageClassName: "efs-sc",
 csi: {
 driver: "efs.csi.aws.com",
 volumeHandle: efsFilesystemId
 }
 }
 },
 { provider: k8sProvider }
);

const pvcEFS = new k8s.core.v1.PersistentVolumeClaim(
 "efs-claim",
 {
 metadata: { name: "efs-claim" },
 spec: {
 accessModes: ["ReadWriteOnce"],
 storageClassName: "efs-sc",
 resources: { requests: { storage: "5Gi" } }
 }
 },
 { provider: k8sProvider }
);

//* STEP 6: Mount the endpoint to pod in EKS cluster

export const newPod = new k8s.core.v1.Pod(
 "efs-app",
 {
 metadata: { name: "efs-app" },
 spec: {
 containers: [
 {
 name: "app",
 image: "centos",
 command: ["/bin/sh"],
 args: [
 "-c",
 "while true; do echo $(date -u) >> /data/out.txt; sleep 5; done"
 ],
 volumeMounts: [
 {
 name: "persistent-storage",
 mountPath: "/data"
 }
 ]
 }
 ],
 volumes: [
 {
 name: "persistent-storage",
 persistentVolumeClaim: { claimName: "efs-claim" }
 }
 ]
 }
 },
 { provider: k8sProvider, dependsOn: pvcEFS }
);

Verify the pod is running and that data is being written into the EFS filesystem using:

$ kubectl exec -ti efs-app -- tail -f /data/out.txt
Mon Jul 8 06:12:00 UTC 2019
Mon Jul 8 06:12:05 UTC 2019
Mon Jul 8 06:12:10 UTC 2019
Mon Jul 8 06:12:15 UTC 2019

This brings us to the end of our solution with Pulumi and AWS EFS on Amazon EKS. For more examples, refer to Pulumi’s open source repository here or refer to my other Kubernetes articles.

Deploy Kubernetes Ingress with AWS ALB & Pulumi

Nishi Davidson — Tue, 09 Jul 2019 00:00:00 +0000

Kubernetes Ingress is an API object that allows you manage external (or) internal HTTP[s] access to Kubernetes services running in a cluster. Amazon Elastic Load Balancing Application Load Balancer (ALB) is a popular AWS service that load balances incoming traffic at the application layer across multiple targets, such as Amazon EC2 instances, in a region. ALB supports multiple features including host or path based routing, TLS (Transport layer security) termination, WebSockets, HTTP/2, AWS WAF (web application firewall) integration, integrated access logs, and health checks.

The AWS ALB Ingress controller is a Kubernetes SIG-AWS subproject - it was the second sub-project added to SIG-AWS after the aws-authenticator subproject. The ALB Ingress controller triggers the creation of an ALB and the necessary supporting AWS resources whenever a Kubernetes user declares an Ingress resource on the cluster. TargetGroups are created for each backend specified in the Ingress resource. Listeners are created for every port specified as Ingress resource annotation. When no port is specified, sensible defaults (80 or 443) are used. Rules are created for each path specified in your ingress resource. This ensures that traffic to a specific path is routed to the correct TargetGroup.

In this post, we will work through a simple example of running ALB based Kubernetes Ingresses with Pulumi EKS, AWS, and AWSX packages.

Step 1: Initialize Pulumi project and stack

Install pulumi CLI and set up your AWS credentials. Initialize a new Pulumi project and Pulumi stack from available programming language templates. We will use the aws-typescript template here and install all library dependencies.

$ brew install pulumi/tap/pulumi # download pulumi CLI
$ mkdir eks-alb-ingress && cd eks-alb-ingress
$ pulumi new aws-typescript
$ npm install --save @pulumi/kubernetes @pulumi/eks
$ ls -la
drwxr-xr-x 10 nishidavidson staff 320 Jun 18 18:22 .
drwxr-xr-x+ 102 nishidavidson staff 3264 Jun 18 18:13 ..
-rw------- 1 nishidavidson staff 21 Jun 18 18:22 .gitignore
-rw-r--r-- 1 nishidavidson staff 32 Jun 18 18:22 Pulumi.dev.yaml
-rw------- 1 nishidavidson staff 91 Jun 18 18:22 Pulumi.yaml
-rw------- 1 nishidavidson staff 273 Jun 18 18:22 index.ts
drwxr-xr-x 95 nishidavidson staff 3040 Jun 18 18:22 node_modules
-rw-r--r-- 1 nishidavidson staff 50650 Jun 18 18:22 package-lock.json
-rw------- 1 nishidavidson staff 228 Jun 18 18:22 package.json
-rw------- 1 nishidavidson staff 522 Jun 18 18:22 tsconfig.json

Step 2: Create an EKS cluster

Once the steps above are complete, we update the typescript code in index.ts file to create an EKS cluster and run pulumi up from the command line:

import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";

const vpc = new awsx.ec2.Vpc("vpc-alb-ingress-eks", {});
const cluster = new eks.Cluster("eks-cluster", {
 vpcId: vpc.id,
 subnetIds: vpc.publicSubnetIds,
 instanceType: "t2.medium",
 version: "1.12",
 nodeRootVolumeSize: 200,
 desiredCapacity: 3,
 maxSize: 4,
 minSize: 3,
 deployDashboard: false,
 vpcCniOptions: {
 warmIpTarget: 4
 }
});

export const clusterName = cluster.eksCluster.name;
export const kubeconfig = cluster.kubeconfig;
export const clusterNodeInstanceRoleName = cluster.instanceRoles.apply(
 roles => roles[0].name
);
export const nodesubnetId = cluster.core.subnetIds;

Configure the Public subnets in the console as defined in this guide.

Step 3: Deploy AWS ALB Ingress Controller

Lets confirm that the EKS cluster is up using the following commands:

$ pulumi stack export kubeconfig > kubeconfig.yaml
$ export KUBECONFIG=kubeconfig.yaml
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
ip-10-10-0-58.ec2.internal Ready <none> 7h8m v1.12.7
ip-10-10-1-167.ec2.internal Ready <none> 7h8m v1.12.7
ip-10-10-1-84.ec2.internal Ready <none> 7h8m v1.12.7

Adequate roles and policies must be configured in AWS and available to the node(s) running the controller. How access is granted is up to you. Some will attach the needed rights to node’s role in AWS. Others will use projects like kube2iam. We attach a minimal IAM policy to the EKS worker nodes and then declare this on the EKS cluster as shown in the code below.

When declaring the ALB Ingress controller we simply re-use the Helm chart as part of the code. There is no need to rewrite all the logic or install Tiller in the EKS cluster. This frees you from thinking about RBAC for Helm, Tiller and the k8s cluster per se'.

With the default “instance mode” Ingress traffic starts from the ALB and reaches the NodePort opened for the service. Traffic is then routed to the container Pods within cluster. This is all encoded using Pulumi libraries below. If you wish to use “ip-mode” with your Ingress such that traffic directly reaches your pods, you will need to modify the alb.ingress.kubernetes.io/target-type annotation when using the helm chart.

Append index.ts file from Step 2 with the code below and run pulumi up:

// STEP 3: Declare the AWS ALB Ingress Controller

// Create IAM Policy for the IngressController called "ingressController-iam-policy” and read the policy ARN.
const ingressControllerPolicy = new aws.iam.Policy(
 "ingressController-iam-policy",
 {
 policy: {
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Action: [
 "acm:DescribeCertificate",
 "acm:ListCertificates",
 "acm:GetCertificate"
 ],
 Resource: "*"
 },
 {
 Effect: "Allow",
 Action: [
 "ec2:AuthorizeSecurityGroupIngress",
 "ec2:CreateSecurityGroup",
 "ec2:CreateTags",
 "ec2:DeleteTags",
 "ec2:DeleteSecurityGroup",
 "ec2:DescribeInstances",
 "ec2:DescribeInstanceStatus",
 "ec2:DescribeSecurityGroups",
 "ec2:DescribeSubnets",
 "ec2:DescribeTags",
 "ec2:DescribeVpcs",
 "ec2:ModifyInstanceAttribute",
 "ec2:ModifyNetworkInterfaceAttribute",
 "ec2:RevokeSecurityGroupIngress"
 ],
 Resource: "*"
 },
 {
 Effect: "Allow",
 Action: [
 "elasticloadbalancing:AddTags",
 "elasticloadbalancing:CreateListener",
 "elasticloadbalancing:CreateLoadBalancer",
 "elasticloadbalancing:CreateRule",
 "elasticloadbalancing:CreateTargetGroup",
 "elasticloadbalancing:DeleteListener",
 "elasticloadbalancing:DeleteLoadBalancer",
 "elasticloadbalancing:DeleteRule",
 "elasticloadbalancing:DeleteTargetGroup",
 "elasticloadbalancing:DeregisterTargets",
 "elasticloadbalancing:DescribeListeners",
 "elasticloadbalancing:DescribeLoadBalancers",
 "elasticloadbalancing:DescribeLoadBalancerAttributes",
 "elasticloadbalancing:DescribeRules",
 "elasticloadbalancing:DescribeSSLPolicies",
 "elasticloadbalancing:DescribeTags",
 "elasticloadbalancing:DescribeTargetGroups",
 "elasticloadbalancing:DescribeTargetGroupAttributes",
 "elasticloadbalancing:DescribeTargetHealth",
 "elasticloadbalancing:ModifyListener",
 "elasticloadbalancing:ModifyLoadBalancerAttributes",
 "elasticloadbalancing:ModifyRule",
 "elasticloadbalancing:ModifyTargetGroup",
 "elasticloadbalancing:ModifyTargetGroupAttributes",
 "elasticloadbalancing:RegisterTargets",
 "elasticloadbalancing:RemoveTags",
 "elasticloadbalancing:SetIpAddressType",
 "elasticloadbalancing:SetSecurityGroups",
 "elasticloadbalancing:SetSubnets",
 "elasticloadbalancing:SetWebACL"
 ],
 Resource: "*"
 },
 {
 Effect: "Allow",
 Action: ["iam:GetServerCertificate", "iam:ListServerCertificates"],
 Resource: "*"
 },
 {
 Effect: "Allow",
 Action: [
 "waf-regional:GetWebACLForResource",
 "waf-regional:GetWebACL",
 "waf-regional:AssociateWebACL",
 "waf-regional:DisassociateWebACL"
 ],
 Resource: "*"
 },
 {
 Effect: "Allow",
 Action: ["tag:GetResources", "tag:TagResources"],
 Resource: "*"
 },
 {
 Effect: "Allow",
 Action: ["waf:GetWebACL"],
 Resource: "*"
 }
 ]
 }
 }
);

// Attach this policy to the NodeInstanceRole of the worker nodes.
export const nodeinstanceRole = new aws.iam.RolePolicyAttachment(
 "eks-NodeInstanceRole-policy-attach",
 {
 policyArn: ingressControllerPolicy.arn,
 role: clusterNodeInstanceRoleName
 }
);

// Declare the ALBIngressController in 1 step with the Helm Chart.
const albingresscntlr = new k8s.helm.v2.Chart(
 "alb",
 {
 chart:
 "http://storage.googleapis.com/kubernetes-charts-incubator/aws-alb-ingress-controller-0.1.9.tgz",
 values: {
 clusterName: clusterName,
 autoDiscoverAwsRegion: "true",
 autoDiscoverAwsVpcID: "true"
 }
 },
 { provider: cluster.provider }
);

Confirm the alb-ingress-controller was created as follows:

$ kubectl get pods -n default | grep alb
alb-aws-alb-ingress-controller-58f44d4bb8lxs6w
$ kubectl logs alb-ingress-controller-58f44d4bb8lxs6w
-------------------------------------------------------------------------------
AWS ALB Ingress controller
Release: v1.1.2
Build: git-cc1c5971
Repository: https://github.com/kubernetes-sigs/aws-alb-ingress-controller.git
-------------------------------------------------------------------------------

Make sure the ingress-controller logs do not show errors about missing subnet tags or missing cluster name before proceeding to Step 4.

Step 4: Deploy Sample Application

The Ingress controller should now be running on the EKS worker nodes. Let’s now create a sample “2048-game” and expose it as an Ingress on our EKS cluster. The code below will let you do so. Append this piece of code into index.ts file from Step 3 and run pulumi up:

function createNewNamespace(name: string): k8s.core.v1.Namespace {
 //Create new namespace
 return new k8s.core.v1.Namespace(
 name,
 { metadata: { name: name } },
 { provider: cluster.provider }
 );
}

// Define the 2048 namespace, deployment, and service.
const nsgame = createNewNamespace("2048-game");

const deploymentgame = new k8s.extensions.v1beta1.Deployment(
 "deployment-game",
 {
 metadata: { name: "deployment-game", namespace: "2048-game" },
 spec: {
 replicas: 5,
 template: {
 metadata: { labels: { app: "2048" } },
 spec: {
 containers: [
 {
 image: "alexwhen/docker-2048",
 imagePullPolicy: "Always",
 name: "2048",
 ports: [{ containerPort: 80 }]
 }
 ]
 }
 }
 }
 },
 { provider: cluster.provider }
);

const servicegame = new k8s.core.v1.Service(
 "service-game",
 {
 metadata: { name: "service-2048", namespace: "2048-game" },
 spec: {
 ports: [{ port: 80, targetPort: 80, protocol: "TCP" }],
 type: "NodePort",
 selector: { app: "2048" }
 }
 },
 { provider: cluster.provider }
);

//declare 2048 ingress
const ingressgame = new k8s.extensions.v1beta1.Ingress(
 "ingress-game",
 {
 metadata: {
 name: "2048-ingress",
 namespace: "2048-game",
 annotations: {
 "kubernetes.io/ingress.class": "alb",
 "alb.ingress.kubernetes.io/scheme": "internet-facing"
 },
 labels: { app: "2048-ingress" }
 },
 spec: {
 rules: [
 {
 http: {
 paths: [
 {
 path: "/*",
 backend: { serviceName: "service-2048", servicePort: 80 }
 }
 ]
 }
 }
 ]
 }
 },
 { provider: cluster.provider }
);

After few seconds, verify the Ingress resource as follows:

$ kubectl get ingress/2048-ingress -n 2048-game
NAME HOSTS ADDRESS PORTS AGE
2048-ingress * DNS-Name-Of-Your-ALB 80 3m

Open a browser. Copy and paste your “DNS-Name-Of-Your-ALB”. You should be to access your newly deployed 2048 game – have fun!

Pulumi is open source and free to use. For more information on our product platform, check out the following resources:

Create Secure Jupyter Notebooks on Kubernetes using Pulumi

Nishi Davidson — Thu, 30 May 2019 00:00:00 +0000

In this post, we will work through an example that shows how to use Pulumi to create Jupyter Notebooks on Kubernetes. Having worked on Kubernetes since 2015, a couple of critical benefits jump out that may resonate with you as well:

You write everything in code - TypeScript in our example here.
You need not initialize Tiller or Helm to work with existing Helm charts like nginx-ingress-controller that we use here.
The security patterns in Helm and Tiller are no longer concerns, rather you get to focus on the RBAC of the actual service which is Jupyter-notebook in this example.
You accomplish more with less YAML and iteratively work towards your use cases.

Prerequisites

Install Pulumi
Install Node.js version 6 or later
Install a package manager for Node.js, such as npm or Yarn
Install Google Cloud SDK
Install Helm and only initialize with helm init —client-only

We will work this example on a GKE cluster so lets first configure Google Cloud Auth:

$ gcloud auth login
$ gcloud config set project <YOUR_GCP_PROJECT_HERE>
$ gcloud auth application-default login

Step 1: Create a Pulumi Project and Stack with a Pulumi TypeScript Template

$ mkdir gke-jupyter-notebook && cd gke-jupyter-notebook
$ pulumi new typescript
$ npm install --save @pulumi/kubernetes @pulumi/gcp

Step 2: Create a GKE Cluster

To create a GKE cluster, simply update the following code in index.ts file and run pulumi up.

import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";
import { readFileSync, fstat } from "fs";

const name = "jupyter-notebook";

/*
 * STEP 2: Create a GKE Cluster
 */
const engineVersion = gcp.container.getEngineVersions().then(v => v.latestMasterVersion);
const cluster = new gcp.container.Cluster(name, {
 initialNodeCount: 2,
 minMasterVersion: engineVersion,
 nodeVersion: engineVersion,
 nodeConfig: {
 machineType: "n1-standard-1",
 oauthScopes: [
 "https://www.googleapis.com/auth/compute",
 "https://www.googleapis.com/auth/devstorage.read_only",
 "https://www.googleapis.com/auth/logging.write",
 "https://www.googleapis.com/auth/monitoring"
 ],
 },
});

// Export the Cluster name
export const clusterName = cluster.name;

// Manufacture a GKE-style kubeconfig.
export const kubeconfig = pulumi.
 all([ cluster.name, cluster.endpoint, cluster.masterAuth ]).
 apply(([ name, endpoint, masterAuth ]) => {
 const context = `${gcp.config.project}_${gcp.config.zone}_${name}`;
 return `apiVersion: v1
clusters:
- cluster:
 certificate-authority-data: ${masterAuth.clusterCaCertificate}
 server: https://${endpoint}
 name: ${context}
contexts:
- context:
 cluster: ${context}
 user: ${context}
 name: ${context}
current-context: ${context}
kind: Config
preferences: {}
users:
- name: ${context}
 user:
 auth-provider:
 config:
 cmd-args: config config-helper --format=json
 cmd-path: gcloud
 expiry-key: '{.credential.token_expiry}'
 token-key: '{.credential.access_token}'
 name: gcp
`;
 });

// Create a Kubernetes provider instance that uses our cluster from above.
const clusterProvider = new k8s.Provider(name, { kubeconfig: kubeconfig });

Step 3: Create a NGINX-Ingress-Controller to Generate Ingresses

You can use the default L7 load balancer in Google Cloud but here we add an NGINX-Ingress-Controller to the cluster. With Pulumi, you write four lines of typescript to reuse the stable NGINX ingress controller helm chart and have the controller, default backends, RBAC, Service account, Config map all running within seconds.

Add the following lines of code in index.ts file and run pulumi up once again.

/*
 * STEP 3: Create NGINX Ingress Controller in GKE
 */

const nginxingresscntlr = new k8s.helm.v2.Chart("nginxingresscontroller", {
 repo: "stable",
 chart: "nginx-ingress",
 version: "0.24.1",
 values: {},
}, { providers: { kubernetes: clusterProvider } });

Step 4: Create a Jupyter Notebook Deployment and Service with Type NodePort

Bringing up Jupyter notebook deployment and service requires adding the following lines of code in index.ts file and running pulumi up to apply the changes.

/*
 * STEP 4: Create Jupyter notebook deployment and service in the GKE cluster
 */

const appName = "jupyter-notebook";
const appLabels = { app: appName };
const jupyterNotebook = new k8s.apps.v1beta1.Deployment(appName, {
 metadata: { name: appName, labels: appLabels },
 spec: {
 selector: { matchLabels: appLabels },
 replicas: 1,
 template: {
 metadata: { labels: appLabels },
 spec: {
 containers: [
 {
 name: appName,
 image: "jupyter/tensorflow-notebook",
 ports: [{ containerPort: 8888 }],
 command: ["start-notebook.sh"],
 args: ["--NotebookApp.token=''"]
 }]
 }
 }
 }
}, { provider: clusterProvider });

const jupyterService = new k8s.core.v1.Service(appName, {
 metadata: {
 name: appName,
 labels: appLabels
 },
 spec: {
 type: "NodePort",
 selector: appLabels,
 ports: [{ protocol: "TCP", nodePort: 30040, port: 8888, targetPort: 8888 }],
 }
}, { provider: clusterProvider });

Step 5: Create a Secret that is used with your Jupyter Notebook Domain Name

We first create a local auth.txt file with the password using the following command:

htpasswd -c auth.txt jupyter

We then read this file synchronously, convert it to base64 and add it as a secret in the GKE cluster. We use this secret as the TLS password to access the Jupyter notebook ingress endpoint accessible from the domain name defined in the host section of the ingress declaration. The annotations in the ingress declarations are required to enable this behavior on the ingress object.

/*
 * STEP 5: Create a secret to enable "basic-auth" for your Jupyter
 * notebook ingress and add it to the ingress declaration in the
 * GKE cluster
 */

const authContents = (readFileSync("<path-to-auth.txt-file>")).toString()

function toBase64(s: string): string {
 return Buffer.from(s).toString("base64");
}

const authContents_base64 = toBase64(authContents);

const jupyternotebooksecret = new k8s.core.v1.Secret("jupyter-notebook-tls", {
 metadata: { name: "basic-auth", namespace: "default" },
 type: "Opaque",
 data: {
 auth: authContents_base64,
 }
})

export const jupyternotebookingress = new k8s.extensions.v1beta1.Ingress(appName, {
 metadata: {
 name: appName,
 labels: appLabels,
 annotations: {
 "kubernetes.io/tls-acme": "true",
 "kubernetes.io/ingress.class": "nginx",
 "nginx.ingress.kubernetes.io/auth-type": "basic",
 "nginx.ingress.kubernetes.io/auth-secret": "basic-auth",
 "nginx.ingress.kubernetes.io/auth-realm": "Authentication Required - jupyter"
 },
 },
 spec: {
 rules: [{
 host: "nishidavidson.com",
 http: {
 paths: [{
 path: "/",
 backend: { serviceName: "jupyter-notebook", servicePort: 8888 }
 }]
 }
 }],
 tls: [{
 secretName: "jupyter-notebook-tls",
 hosts: ["nishidavidson.com"]
 }],
 }
}, { provider: clusterProvider });

Once complete, you will see the GKE cluster components show up as follows:

Open a browser to access the jupyter-notebook Service using the domain name declared in the host section of your ingress code above. In our example, I used the domain name nishidavidson.com.

When you open your DNS, you will be asked for the “username: jupyter” and “password” for the secret you created in auth.txt file as shown below:

As soon as the password is accepted, You should now be able to go to your website from anywhere in the world and access your password-protected Jupyter notebook running on GKE on secure SSL connection.

Success! We worked through a simple example of creating a GKE cluster, an NGINX ingress controller and stood up our password protected Jupyter notebook Ingress, Service, and Deployment with a simple secret for authentication.

You can get started today using more solutions on Kubernetes from our Pulumi examples repository.

Continuous Delivery with GitLab and Pulumi on Amazon EKS

Nishi Davidson — Wed, 22 May 2019 00:00:00 +0000

In this blog, we will work through an example that shows how to use Pulumi to enable GitLab-based continuous delivery with your Kubernetes workloads on Amazon EKS. This integration will work just as seamlessly for any Kubernetes cluster, including Azure AKS or Google GKE, using the relevant Pulumi libraries for Azure and Google Cloud.

Prerequisites

An account on https://app.pulumi.com with an organization.
The latest pulumi CLI. Installation instructions are here.
A bare repository. Set the remote URL to be your GitLab project.

Concepts in Pulumi

Organization of Pulumi Projects and Pulumi Stacks

All users in the Pulumi service will start with the hierarchy of an organization. This can be a specific GitHub, GitLab or Atlassian organization or your solo organization. Inside each organization, users create Pulumi projects and stacks.

Pulumi projects and stacks are flexible to accommodate the diverse needs across teams, applications, and infrastructure scenarios. Just like Git repos that work with varying approaches Pulumi projects and stacks allow you to organize your code within them. Immediate options include:

Monolithic project/stack structure: A single project defines the infrastructure and application resources for an entire vertical service as represented in the image below:

Micro-stacks project/stack structure: A project broken into separately managed smaller projects, often across different dimensions as represented in the image below:

Working with Inter-Stack Dependencies with the latter option is more suited in a production setup giving users more flexibility and boundaries between their teams. We will use this structure in our example below. For more information on Pulumi projects and stacks, please refer to our organizaing documentation.

Use Tags to group Pulumi Stacks as Environments:

Pulumi Stacks have associated metadata in the form of key/value tags.
You can assign custom tags to stacks (when logged into the Pulumi Service backend to customize how stacks are listed in the Pulumi Service.
- In our example below we have two environments prod and dev.
- To group stacks by environment we assign custom tags environment: prod and environment: dev to the respective stacks
- In the Pulumi Service, you’ll be able to group stacks by tag: environment:dev and tag: environment:prod.

Please read more about managing stack tags in Pulumi.

Let’s now work through our example with GitLab Pipelines.

Please replace the highlighted <org-name-in-pulumi> below with your Pulumi organization name to run through the steps successfully.

GitLab Pipeline by Environment

We created a GitLab Group called pulumi-gitlab.
We created three GitLab projects called sample-iam, sample-eks and sample-k8sapp. These projects match the project names in the Pulumi Service.
We have two pipelines: environment:dev and environment:prod. In the two pipelines, we have a total of six pulumi stacks:
- <org-name-in-pulumi>/sample-iam/dev, and <org-name-in-pulumi>/sample-iam/prod.
- <org-name-in-pulumi>/sample-eks/dev and <org-name-in-pulumi>/sample-eks/prod.
- <org-name-in-pulumi>/sample-k8sapp/dev and <org-name-in-pulumi>/sample-k8sapp/prod.
<org-name-in-pulumi>/sample-iam/dev stack will trigger the downstream stack <org-name-in-pulumi>/sample-eks/dev provided the cycle of pulumi preview and pulumi deploy completes without any failure.

Similarly, <org-name-in-pulumi>/sample-eks/dev will trigger the downstream stack <org-name-in-pulumi>/sample-k8sapp/dev provided the cycle of pulumi preview and pulumi deploy completes without any failure.
To use Pulumi within GitLab CI, there are a few environment variables you’ll need to set for each build.
- The first is PULUMI_ACCESS_TOKEN, which is required to authenticate with pulumi.com in order to perform the preview or update. You can create a new Pulumi access token specifically for your CI/CD job on your Pulumi Account page.
- Next, you will also need to set environment variables specific to your cloud resource provider. For example, if your stack is managing resources on AWS, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

Create Pulumi stacks and push the files to your GitLab project

If you run pulumi from any branch other than the master branch, you will hit an error that the PULUMI_ACCESS_TOKEN environment variable cannot be accessed. You can fix this by specifying a wildcard regex to allow specific branches to be able to access the secret environment variables. Please refer to the GitLab documentation to understand this better.

First we set up three Pulumi stacks: sample-iam; sample-eks and sample-k8sapp with group stack tag: environment:dev

Step 1: Create the pulumi stack “sample-IAM” and set stack tag “key:value” = “environment:dev”. We update the index.ts file with the relevant code block as shown below and run pulumi up.

$ pulumi new aws-typescript --dir <org-name-in-pulumi>/sample-iam/dev
$ cd <org-name-in-pulumi>/sample-iam/dev

Let’s run pulumi up with the following index.ts file.

import * as aws from "@pulumi/aws";
function createIAMRole(name: string): aws.iam.Role {
 // Create an IAM Role
 return new aws.iam.Role(`${name}`, {
 assumeRolePolicy: `{
 "Version": "2012-10-17",
 "Statement":[
 {
 "Sid": "",
 "Effect": "Allow",
 "Principal": {
 "AWS": "arn:aws:iam::153052954103:root"
 },
 "Action": "sts:AssumeRole"
 }
 ]
 }`,
 tags: {
 "clusterAccess": `${name}-usr`,
 },
 });
}

// Administer automation role for use in pipelines, e.g. gitlab CI, Teamcity, etc.
export const automationRole = createIAMRole("automationRole");
export const automationRoleArn = automationRole.arn;

We then group the stack by initializing it with a new stack tag “key:value” = “environment:prod” and run pulumi up with the same index.ts file.

$ pulumi up

# Initialize new pulumi stack in the format:
# pulumi stack init <org-name-in-pulumi>/<project>/<stack>
$ pulumi stack init <org-name-in-pulumi>/sample-iam/prod
$ pulumi stack tag set environment prod
$ pulumi up

Step 2: Create the pulumi stack sample-eks and set stack tag “key:value” = “environment:dev”.

$ pulumi new aws-typescript --dir <org-name-in-pulumi>/sample-eks/dev
$ cd <org-name-in-pulumi>/sample-eks/dev

Update the index.ts file with the relevant code block as shown below:

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

const env = pulumi.getStack();
const iamstack = new pulumi.StackReference(`<org-name-in-pulumi>/sample-iam/${env}`);

const automationRoleArn = iamstack.getOutput("automationRoleArn")

/* * Single step deployment of EKS cluster with the most important variables and simple function to create two namespaces */

const vpc = new awsx.ec2.Vpc("vpc", {});
const cluster = new eks.Cluster("eks-cluster", {
 vpcId : vpc.id,
 subnetIds : vpc.publicSubnetIds,
 instanceType : "t2.medium",
 nodeRootVolumeSize: 200,
 desiredCapacity : 1,
 maxSize : 2,
 minSize : 1,
 deployDashboard : false,
 vpcCniOptions : {
 warmIpTarget : 4,
 },
 roleMappings : [
 // Map IAM role arn "automationRoleArn" to the k8s user with name "automation-usr", e.g. gitlab CI
 {
 groups : ["pulumi:automation-grp"],
 roleArn : automationRoleArn,
 username : "pulumi:automation-usr",
 },
 ],
});

export const clusterName = cluster.eksCluster.name;

/* * Single Step deployment of k8s RBAC configuration */

new k8s.rbac.v1.Role("automationRole", {
 metadata: {
 name: "automationRole",
 namespace: "automation",
 },
 rules: [{
 apiGroups: ["*"],
 resources: ["*"],
 verbs: ["*"],
 }]
}, {provider: cluster.provider});

new k8s.rbac.v1.RoleBinding("automation-binding", {
 metadata: {
 name: "automation-binding",
 namespace: "automation",
 },
 subjects: [{
 kind: "User",
 name: "pulumi:automation-usr",
 apiGroup: "rbac.authorization.k8s.io",
 }],
 roleRef: {
 kind: "Role",
 name: "automationRole",
 apiGroup: "rbac.authorization.k8s.io",
 },
}, {provider: cluster.provider});

export const kubeconfig = cluster.kubeconfig.apply(JSON.stringify)

Let’s download the additional npm packages for EKS and Kubernetes and run pulumi up and initialize a new stack tag “key:value” = “environment:prod” to run pulumi up with the same index.ts file.

$ npm install --save @pulumi/eks @pulumi/kubernetes
$ pulumi up
# Initialize new pulumi stack in the format:
# pulumi stack init <org-name-in-pulumi>/<project>/<stack>
$ pulumi stack init <org-name-in-pulumi>/sample-eks/prod
$ pulumi stack tag set environment prod
$ pulumi up

Step 3: Create the pulumi stack “sample-k8sapp” and set stack tag “key:value” = “environment:dev”. We will then update the index.ts file with the relevant code block as shown below:

$ pulumi new aws-typescript --dir <org-name-in-pulumi>/sample-k8sapp/dev
$ cd <org-name-in-pulumi>/sample-k8sapp/dev

import * as aws from "@pulumi/aws";
import * as docker from "@pulumi/docker";
import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

const env = pulumi.getStack();
const eksCluster = new pulumi.StackReference(`<org-name-in-pulumi>/sample-eks/${env}`);

const kubeconfig = eksCluster.getOutput("kubeconfig");

const k8sProvider = new k8s.Provider("eks-cluster", {
 kubeconfig: kubeconfig,
 });

/* * Single step deployment of one docker container in ECR */

function getImageRegistry(repo: aws.ecr.Repository) {
 return repo.registryId.apply(async registryId => {
 if (!registryId) {
 throw new Error("Expected registry ID to be defined during push");
 }
 const credentials = await aws.ecr.getCredentials({ registryId: registryId });
 const decodedCredentials = Buffer.from(credentials.authorizationToken, "base64").toString();
 const [username, password] = decodedCredentials.split(":");
 if (!password || !username) {
 throw new Error("Invalid credentials");
 }
 return {
 server: credentials.proxyEndpoint,
 username: username,
 password: password,
 };
 });
}

const ecr1 = new aws.ecr.Repository("breathe");
const image1 = new docker.Image("breathe", {
 imageName: ecr1.repositoryUrl,
 build: {
 context: "./app",
 cacheFrom: true,
 },
 registry: getImageRegistry(ecr1),
});

// Declare the docker container based deployment
const appLabels = { app: appName };
const breathecontainer = new k8s.apps.v1beta1.Deployment(appName, {
 spec: {
 selector: { matchLabels: appLabels },
 replicas: 1,
 template: {
 metadata: { labels: appLabels },
 spec: { containers: [{ name: appName, image: image1.imageName }] }
 }
 },
 }, { provider: k8sProvider });

We download the additional npm packages for EKS and Kubernetes and run pulumi up and group a new stack by initializing it with a new stack tag “key:value” = “environment:prod” and run pulumi up with the same index.ts file.

$ npm install --save @pulumi/kubernetes @pulumi/docker
$ pulumi up

# Initialize new pulumi stack
$ pulumi stack init <org-name-in-pulumi>/sample-eks/prod
$ pulumi stack tag set environment prod
$ pulumi up

Using GitLab Pipelines with the six Pulumi stacks in `environment:dev` and `environment:prod`

GitLab pipelines are configured using .gitlab-ci.yml files in the root of each repository. GitLab Silver and above is capable of running pipelines that cross project boundaries, so we will be using that to construct our pipeline.

All three .gitlab-ci.yml files that we use are very similar in structure. The base one,sample-iam, looks like this:

image:
 name: pulumi/pulumi:v0.17.10
 entrypoint:
 - '/usr/bin/env'
 - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'

stages:
 - preview
 - update
 - downstream

Pulumi Preview:
 stage: preview
 script:
 - npm ci
 - pulumi stack select pulumi/sample-iam/$DEPLOY_ENVIRONMENT
 - pulumi preview

Pulumi Update:
 stage: update
 script:
 - npm ci
 - pulumi stack select pulumi/sample-iam/$DEPLOY_ENVIRONMENT
 - pulumi update --skip-preview

Update EKS:
 stage: downstream
 trigger: pulumi-gitlab/sample-eks

This file describes a three-stage pipeline for the sample-iam project:

First, we run a preview for the requested deployment environment, failing the pipeline if the preview fails.
If the preview was successful, we run pulumi update, which deploys the IAM changes.
Finally, we trigger the pipeline in pulumi-gilab/sample-eks, which triggers the next pipeline in our pipeline daisy chain illustrated in the above image.

Despite being powerful, conceptually this setup is quite simple and doesn’t require much code to get right.

Upon a successful update, each tier’s pipeline will trigger a pipeline for the tiers that depend on it. Pulumi’s StackReference feature ensures that the dependent tiers receive new copies of the outputs exported from the IAM stack, so the deployment flows naturally through the pipeline!

This brings us to the end of our CD solution with Pulumi and GitLab on Amazon EKS. For more examples, refer to the Pulumi examples repository. Refer to my previous post on Amazon EKS and k8s RBAC in Pulumi.

Create AKS Clusters with Monitoring and Logging

Nishi Davidson — Wed, 08 May 2019 00:00:00 +0000

Pulumi-Azure open source package can be used to create Azure Kubernetes Service (AKS) clusters with AD service principals tied to the cluster. Monitoring and logging can also be enabled by default in simple steps with Pulumi for core AKS components. This article will show you how to write this as a simple example using Pulumi SDKs.

Prerequisites

Install pulumi CLI and set up your Azure credentials
Initialize a new Pulumi project from available templates. We use ****azure-typescript**** template here to install all dependencies and save the configuration.

Step 1: Export required configurations for the AKS stack

import * as azure from "@pulumi/azure";
import * as azuread from "@pulumi/azuread";
import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

// Step 1: Parse and export configuration variables for the AKS stack.
const config = new pulumi.Config();
const password = config.require("password");
const location = config.get("location") || "East US";
const nodeCount = config.getNumber("nodeCount") || 2;
const nodeSize = config.get("nodeSize") || "Standard_D2_v2";
const sshPublicKey = config.require("sshPublicKey");

const resourceGroup = new azure.core.ResourceGroup("aks", { location });
const loganalytics = new azure.operationalinsights.AnalyticsWorkspace("aksloganalytics", {
 resourceGroupName: resourceGroup.name,
 location: resourceGroup.location,
 sku: "PerGB2018",
 retentionInDays: 30,
});

Step 2: Create the AD Service principal for the AKS cluster

When you register an Azure AD application in the Azure portal, two objects are created in your Azure AD tenant: a) An application object, and b) A service principal object. To access resources that are secured by an Azure AD tenant, the entity that requires access must be represented by a security principal that defines the access policy and permissions for the user/application in the Azure AD tenant. This is true for both users (user principal) and applications (service principal). This enables core features such as authentication of the user/application during sign-in, and authorization during resource access.

In our example we create the Service principal which is a required in the successful creation of the AKS cluster. If you see that the Service Principal Id was not found: “Service principal clientID: xxxxx not found”, simply re-run pulumi updateto proceed.

// Step 2: Create the AD service principal for the k8s cluster.
const adApp = new azuread.Application("aks");
const adSp = new azuread.ServicePrincipal("aksSp", { applicationId: adApp.applicationId });
const adSpPassword = new azuread.ServicePrincipalPassword("aksSpPassword", {
 servicePrincipalId: adSp.id,
 value: password,
 endDate: "2099-01-01T00:00:00Z",
});

Step 3: Create the AKS cluster

Now let’s add the logic to create the AKS cluster with the Service principal.

// Step 3: This step creates an AKS cluster.
const k8sCluster = new azure.containerservice.KubernetesCluster("aksCluster", {
 resourceGroupName: resourceGroup.name,
 location: location,
 defaultNodePool: {
 name: "aksagentpool",
 nodeCount: nodeCount,
 vmSize: nodeSize,
 },
 dnsPrefix: `${pulumi.getStack()}-kube`,
 linuxProfile: {
 adminUsername: "aksuser",
 sshKey: { keyData: sshPublicKey},
 },
 servicePrincipal: {
 clientId: applicationId,
 clientSecret: adSpPassword.value,
 },
 addonProfile: {
 omsAgent: {
 enabled: true,
 logAnalyticsWorkspaceId: loganalytics.id,
 },
 },
});

// Expose a k8s provider instance using our custom cluster instance.
const k8sProvider = new k8s.Provider("aksK8s", {
 kubeconfig: k8sCluster.kubeConfigRaw,
});

// Export the kubeconfig
export const kubeconfig = k8sCluster.kubeConfigRaw;

Step 4: Enable default monitoring and logging for the AKS cluster

Log queries are used to retrieve any log data from Azure Monitor. Whether you’re analyzing data in the portal, or configuring an alert rule to be notified of a particular condition, you will use a query to specify the data you want.

We will now enable monitoring and logging by default for the cluster and run pulumi up

// Step 4: Enables the Monitoring Diagonostic control plane component logs and AllMetrics
const azMonitoringDiagnostic = new azure.monitoring.DiagnosticSetting("aks", {
 logAnalyticsWorkspaceId: loganalytics.id,
 targetResourceId: k8sCluster.id,
 logs: [{
 category: "kube-apiserver",
 enabled : true,

 retentionPolicy: {
 enabled: true,
 },
 }],
 metrics: [{
 category: "AllMetrics",

 retentionPolicy: {
 enabled: true,
 },
 }],
});

If you run pulumi up with the entire index.ts file as defined here you will see an AKS cluster get created with monitoring and logging enabled.

Step 5: Check if everything worked

a) Check with Pulumi CLI and kubectl

$ pulumi stack output kubeconfig > kubeconfig.yaml
$ kubectl config current-context
akscluster1321f898

b) Check Pulumi Service Console for the resource creation on the stack you just initialized: d-nishi/azure-mean/dev.

c) Check your Azure portal for the cluster-id and to find all the metrics (Insights) and logs (Logs) show up as follows:

Summary

We discussed how to set up AKS clusters with monitoring and logging in simple steps with Pulumi packages. You can find the complete pulumi code for our example here. For more examples visit our GitHub examples repo.

If you want to additionally look at how to integrate Azure DevOps with Pulumi, check out our other post CI/CD made easy with Pulumi and Azure Pipelines.

Kubernetes RBAC in AWS EKS with open source Pulumi packages

Nishi Davidson — Wed, 24 Apr 2019 00:00:00 +0000

Check out version 3.0 of the Pulumi EKS Provider.

One of the most common areas Kubernetes operators struggle with in production involves creating and managing role-based access control (RBAC). This is so daunting that RBAC is often not implemented, or implemented halfway, or the configuration becomes impossible to maintain.

Fortunately, Pulumi makes RBAC on Kubernetes so easy that you’ll never create an insecure cluster again. In this post, we will contrast the traditional way of working with RBAC on EKS with using Pulumi.

Here are a few highlights:

NO MORE YAMLs! Configuring YAMLs, operators or custom resources is now a thing in the past! You use TypeScript or JavaScript to program directly with our cloud SDK and connect all cloud services to your Kubernetes services with a simple reference to the object in your program.
INCREASED DEVELOPMENT VELOCITY: You intuitively program Kubernetes objects with our SDK abstractions using minimal amount of code within hours instead of months. You “autocomplete” AWS, EKS, Kubernetes specifications within your IDE without understanding the entire API.
EASY UPDATES: Changing a roleRef in a RoleBinding, on one or multiple clusters involves updating your TypeScript file index.ts and running pulumi up. The Pulumi Service allows you to share your stack with your team in your GitHub, GitLab, or Atlassian-based organization.
WORKFLOW AUTOMATION FOR RBAC AT SCALE: You can delete or update multiple RoleBindings or Roles from your Pulumi stack source code. As you commit these changes to your repository, you can plan automated triggers that validate such changes as part of your CI/CD flow, whether you use Travis, CircleCI, AzureDevOps and more. Pulumi even has a GitHub Application for surfacing results within pull requests.

Prerequisites to work with Pulumi

Install pulumi CLI and set up your AWS credentials. Initialize a new Pulumi project from available templates. We use aws-typescript template here to install all dependencies and save the configuration.

$ brew install pulumi/tap/pulumi # download pulumi CLI
$ mkdir eks-rbac && cd eks-rbac
$ pulumi new aws-typescript
$ ls -l
-rw-r--r-- 1 nishidavidson staff 32 Apr 18 14:49 Pulumi.dev.yaml
-rw------- 1 nishidavidson staff 84 Apr 18 14:48 Pulumi.yaml
-rw------- 1 nishidavidson staff 273 Apr 18 14:48 index.ts
drwxr-xr-x 92 nishidavidson staff 2944 Apr 18 14:49 node_modules
-rw-r--r-- 1 nishidavidson staff 48352 Apr 18 14:49 package-lock.json
-rw------- 1 nishidavidson staff 228 Apr 18 14:48 package.json
-rw------- 1 nishidavidson staff 522 Apr 18 14:48 tsconfig.json

With Pulumi, you will modify and update the default index.ts file with AWS and EKS resource variable declarations. We show you how to add this code as we contrast Pulumi’s approach with the sequential traditional approach in the steps below. In the end, you will do a one-time run of pulumi up and watch all the steps in the Traditional Way come alive simultaneously.

Step 1: Create three IAM Roles with a Trust-policy to map to Amazon EKS RBAC.

The Traditional-approach:

You sequentially create three IAM roles (clusterAdminRole; AutomationRole; EnvProdRole) with aws command line tool as shown below:

$ aws iam create-role --role-name clusterAdminRole --assume-role-policy-document file://Role-Trust-Policy.json
 
{
    "Role": {
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Action": "sts:AssumeRole",
                    "Principal": {
                        "AWS": "arn:aws:iam::xxxxxxxxxxxx:root"
                    },
                    "Effect": "Allow",
                    "Sid": ""
                }
            ]
        },
        "RoleId": "AROASHIVKXX3SFFMUUEU6",
        "CreateDate": "2019-04-17T17:43:03Z",
        "RoleName": "clusterAdminRole",
        "Path": "/",
        "Arn": "arn:aws:iam::xxxxxxxxxxxx:role/clusterAdminRole"
    }
}
 
$ cat Role-Trust-Policy.json
{
    "Version": "2012-10-17",
    "Statement": [
        {
          "Effect": "Allow",
          "Action": "sts:AssumeRole",
          "Resource": "*"
        }
    ]
}

The Pulumi-approach:

You update the default index.ts file in your source code editor such as VSCode as follows:

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";

/*
 * 1) Single step deployment of three IAM Roles
 */

function createIAMRole(name: string): aws.iam.Role {
    // Create an IAM Role...
    return new aws.iam.Role(`${name}`, {
        assumeRolePolicy: `{
            "Version": "2012-10-17",
            "Statement":[
              {
                "Sid": "",
                "Effect": "Allow",
                "Principal": {
                  "AWS": "arn:aws:iam::153052954103:root"
                },
                "Action": "sts:AssumeRole"
              }
            ]
           }
        `,
          tags: {
              "clusterAccess": `${name}-usr`,
          },
        });
    };
}

// Administrator AWS IAM clusterAdminRole with full access to all AWS resources
const clusterAdminRole = createIAMRole("clusterAdminRole");

// Administer Automation role for use in pipelines, e.g. gitlab CI, Teamcity, etc.
const AutomationRole = createIAMRole("AutomationRole");

// Administer Prod role for use in Prod environment
const EnvProdRole = createIAMRole("EnvProdRole");

Step 2: Create one EKS cluster. Validate cluster creation. Add the namespaces you need.

The Traditional-approach:

You go through the steps below to validate the cluster and k8s resource deployment based on your tool chain and your understanding of Kubernetes:

$ eksctl create cluster eks-nd-test
 
$ kubectl get no
NAME STATUS ROLES AGE VERSION
ip-192-168-41-125.us-east-2.compute.internal Ready <none> 11h v1.11.5
ip-192-168-5-250.us-east-2.compute.internal Ready <none> 11h v1.11.5
 
$ kubectl create -f automation-ns.yaml && kubectl create -f prod-ns.yaml
 
$ cat automation-ns.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: automation
  labels:
    name: automation

The Pulumi-approach:

You use our API docs and your source-code editor to autocomplete the default index.ts file.

/*
 * 2) Single step deployment of EKS cluster with the most important variables and a Simple Function to create namespaces
 * automation and prod
 */

const vpc = new awsx.ec2.Vpc("vpc", {});
const cluster = new eks.Cluster("eks-cluster", {
  vpcId : vpc.id,
  subnetIds : vpc.publicSubnetIds,
  instanceType : "t2.medium",
  nodeRootVolumeSize: 200,
  desiredCapacity : 1,
  maxSize : 2,
  minSize : 1,
  deployDashboard : false,
  vpcCniOptions : {
    warmIpTarget : 4,
  },
  roleMappings : [
    // Provides full administrator cluster access to the k8s cluster
    {
      groups : ["system:masters"],
      roleArn : clusterAdminRole.arn,
      username : "pulumi:admin-usr",
    },
    // Map IAM role arn "AutomationRoleArn" to the k8s user with name "automation-usr", e.g. gitlab CI
    {
      groups : ["pulumi:automation-grp"],
      roleArn : AutomationRole.arn,
      username : "pulumi:automation-usr",
    },
    // Map IAM role arn "EnvProdRoleArn" to the k8s user with name "prod-usr"
    {
      groups : ["pulumi:prod-grp"],
      roleArn : EnvProdRole.arn,
      username : "pulumi:prod-usr",
    },
  ],
});

export const clusterName = cluster.eksCluster.name;

function createNewNamespace(name: string): k8s.core.v1.Namespace {
  // Create new namespace
  return new k8s.core.v1.Namespace(name, { metadata: { name: name } }, { provider: cluster.provider });
}

// Declare namespaces automation and prod.
const automation = createNewNamespace("automation");
const prod = createNewNamespace("prod");

Step 3: Understand Kubernetes RBAC. Declare the Kubernetes objects on the EKS cluster.

The Kubernetes RBAC API declares four top-level types that can be defined as YAMLs syntaxes: a) Role - represents a set of additive rules within a namespace; b) RoleBinding - grants namespace-wide access to k8s subjects and resources; c) ClusterRole - represents a set of additive rules within the cluster; d) ClusterRoleBinding - grants cluster-wide access to k8s subjects and resources.

The Traditional-approach:

You define three k8s users with different privileges in your cluster and test them sequentially: User type1 called pulumi:admin-usr for users have cluster admin rights

 $ cat user1.yaml
 kind: ClusterRole
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
  name: ClusterAdminRole
  # no namespace needed
 rules:
 - apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]
  
 ---
 kind: ClusterRoleBinding
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
   name: cluster-admin-binding
 subjects:
 - kind: User
  name: "pulumi:admin-usr"
  apiGroup: rbac.authorization.k8s.io
 roleRef:
  kind: ClusterRole
  name: ClusterAdminRole
  apiGroup: rbac.authorization.k8s.io`

User type2 called pulumi:automation-usr for users that have permissions to all k8s resources in namespace automation. An e.g would be your CI/CD pipeline

$ cat user2.yaml
 
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: AutomationRole
  namespace: automation
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]
 
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: automation-binding
  namespace: automation
subjects:
- kind: User
  name: "pulumi:automation-usr"
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: AutomationRole
  apiGroup: rbac.authorization.k8s.io

User type 3 called prod-usr for users that have read access to all k8s resources in the namespace prod

$ cat user3.yaml
 
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: EnvProdRole
  namespace: prod
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["get", "list", "watch"]
 
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: env-prod-binding
  namespace: prod
subjects:
- kind: User
  name: "pulumi:prod-usr"
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: EnvProdRole
  apiGroup: rbac.authorization.k8s.io`
$ kubectl apply -f user1.yaml && kubectl apply -f user2.yaml && kubectl apply -f user3.yaml`
 
clusterrole.rbac.authorization.k8s.io/ClusterAdminRole created
clusterrolebinding.rbac.authorization.k8s.io/cluster-admin-binding created
role.rbac.authorization.k8s.io/AutomationRole created
rolebinding.rbac.authorization.k8s.io/automation created
role.rbac.authorization.k8s.io/EnvProdRole created
rolebinding.rbac.authorization.k8s.io/env-prod-binding created

The Pulumi-approach:

Update your index.ts file with more code as follows:

/*
 * 3) Single Step deployment of k8s RBAC configuration for user1, user2 and user3 per our example
 */

// Grant cluster admin access to all admins with k8s ClusterRole and ClusterRoleBinding
new k8s.rbac.v1.ClusterRole("clusterAdminRole", {
  metadata: {
    name: "clusterAdminRole",
  },
  rules: [{
    apiGroups: ["*"],
    resources: ["*"],
    verbs: ["*"],
  }]
}, {provider: cluster.provider});

new k8s.rbac.v1.ClusterRoleBinding("cluster-admin-binding", {
  metadata: {
    name: "cluster-admin-binding",
  },
  subjects: [{
     kind: "User",
     name: "pulumi:admin-usr",
  }],
  roleRef: {
    kind: "ClusterRole",
    name: "clusterAdminRole",
    apiGroup: "rbac.authorization.k8s.io",
  },
}, {provider: cluster.provider});

// User2 called automation-usr for users that have permissions to all k8s resources in the namespace automation
new k8s.rbac.v1.Role("AutomationRole", {
  metadata: {
    name: "AutomationRole",
    namespace: "automation",
  },
  rules: [{
    apiGroups: ["*"],
    resources: ["*"],
    verbs: ["*"],
  }]
}, {provider: cluster.provider});
  
new k8s.rbac.v1.RoleBinding("automation-binding", {
  metadata: {
    name: "automation-binding",
    namespace: "automation",
  },
  subjects: [{
     kind: "User",
     name: "pulumi:automation-usr",
     apiGroup: "rbac.authorization.k8s.io",
  }],
  roleRef: {
    kind: "Role",
    name: "AutomationRole",
    apiGroup: "rbac.authorization.k8s.io",
  },
}, {provider: cluster.provider});

// User3 called prod-usr for users that have read access to all k8s resources in the namespace env-prod
new k8s.rbac.v1.Role("EnvProdRole", {
  metadata: {
    name: "EnvProdRole",
    namespace: "prod",
  },
  rules: [{
    apiGroups: ["*"],
    resources: ["*"],
    verbs: ["get", "watch", "list"],
  }],
}, {provider: cluster.provider});
  
new k8s.rbac.v1.RoleBinding("env-prod-binding", {
  metadata: {
    name: "env-prod-binding",
    namespace: "prod",
  },
  subjects: [{
     kind: "User",
     name: "pulumi:prod-usr",
     apiGroup: "rbac.authorization.k8s.io",
    }],
  roleRef: {
    kind: "Role",
    name: "EnvProdRole",
    apiGroup: "rbac.authorization.k8s.io",
  },
}, {provider: cluster.provider});

export const kubeconfig = cluster.kubeconfig

Step 4: Update aws-iam-authenticator ConfigMap to map the IAM Roles to the Kubernetes usernames.

The Traditional-approach:

You get the three IAM role arns for clusterAdminRole, AutomationRole, EnvProdRole and update the configmap with k8s usernames pulumi:admin-usr, pulumi:automation-usr and pulumi:prod-usr.

mapRoles:
----
- groups:
  - system:masters
  rolearn: arn:aws:iam::XXXXXXXXXXXX:role/clusterAdminRole
  username: pulumi:admin-usr
- groups:
  rolearn: arn:aws:iam::XXXXXXXXXXXX:role/AutomationRole
  username: pulumi:automation-usr
- groups:
  rolearn: arn:aws:iam::XXXXXXXXXXXX:role/EnvProdRole
  username: pulumi:prod-usr
- groups:
  - system:bootstrappers
  - system:nodes
  rolearn: arn:aws:iam::XXXXXXXXXXXX:role/eksctl-eks-rbac-nd-test-nodegroup-NodeInstanceRole-NP542EG8JX8U
  username: system:node:`

The Pulumi-approach:

This step is not required as you have already updated the EKS ConfigMap at cluster creation time in STEP 2 using “RoleMappings”. Simply run pulumi up with the full index.ts file. Watch all your components come alive simultaneously. Setting up RBAC on one EKS cluster is a long convoluted sequential process that requires multiple validations along the way. Imagine the complexity involved when working with multiple tools for an environment that requires multiple groups with many users, namespaces, and clusters.

Testing the Pulumi approach worked

Make sure you run pulumi up with this index.ts file.

$ pulumi stack output kubeconfig | jq > kubeconfig.yaml
$ export KUBECONFIG = kubeconfig.yaml

Assume the IAM role AutomationRole with access to all Kubernetes resources in namespace automation and test if the permissions work.

 "users": [
    {
      "name": "aws",
      "user": {
        "exec": {
          "apiVersion": "client.authentication.k8s.io/v1alpha1",
          "args": [
            "token",
            "-i",
            "eks-cluster-eksCluster-196b0de",
            "-r",
            "arn:aws:iam::xxxxxxxxxxxx:role/AutomationRole"
          ],
          "command": "aws-iam-authenticator"
        }
      }
    }
  ]
}
 
$ kubectl get po --namespace=automation
No resources found.
 
$ kubectl get po --namespace=prod
Error from server (Forbidden): pods is forbidden: User "pulumi: automation-usr" cannot list resource "pods" in API group "" in the namespace "prod"

Upon assuming the IAM role AutomationRole which is mapped to Kubernernetes username pulumi:automation-usr in the EKS cluster configmap, you are only restricted to the resources and verbs allowed in the namespace “automation” and not in namespace “prod”.

Next Step

In this post, we discussed how setting up Kubernetes RBAC with Pulumi is simple, comprehensive, non-sequential and part of your everyday programming experience. You can find the complete pulumi code for our example and try it out yourself.

Pulumi is open source and free to use. For more examples, visit our GitHub examples page here. To learn more about Pulumi and how to manage Kubernetes through code, have a look at our “Get Started with Kubernetes” guide.

Pulumi Blog: Nishi Davidson

Persisting Kubernetes workloads with Amazon EFS CSI volumes

Step 1: Initialize Pulumi Project and Stack for your organization:

Step 2: Create the EKS cluster, EFS endpoint and MountTargets:

Step 3: Create the CSI Driver Node and Controller Components

Step 4: Deploy a sample app with the EFS volume mounts:

Deploy Kubernetes Ingress with AWS ALB & Pulumi

Step 1: Initialize Pulumi project and stack

Step 2: Create an EKS cluster

Step 3: Deploy AWS ALB Ingress Controller

Step 4: Deploy Sample Application

Create Secure Jupyter Notebooks on Kubernetes using Pulumi

Prerequisites

Step 1: Create a Pulumi Project and Stack with a Pulumi TypeScript Template

Step 2: Create a GKE Cluster

Step 3: Create a NGINX-Ingress-Controller to Generate Ingresses

Step 4: Create a Jupyter Notebook Deployment and Service with Type NodePort

Step 5: Create a Secret that is used with your Jupyter Notebook Domain Name

Continuous Delivery with GitLab and Pulumi on Amazon EKS

Prerequisites

Concepts in Pulumi

Organization of Pulumi Projects and Pulumi Stacks

Use Tags to group Pulumi Stacks as Environments:

GitLab Pipeline by Environment

Create Pulumi stacks and push the files to your GitLab project

Using GitLab Pipelines with the six Pulumi stacks in environment:dev and environment:prod

Create AKS Clusters with Monitoring and Logging

Prerequisites

Step 1: Export required configurations for the AKS stack

Step 2: Create the AD Service principal for the AKS cluster

Step 3: Create the AKS cluster

Step 4: Enable default monitoring and logging for the AKS cluster

Step 5: Check if everything worked

Summary

Kubernetes RBAC in AWS EKS with open source Pulumi packages

Prerequisites to work with Pulumi

Step 1: Create three IAM Roles with a Trust-policy to map to Amazon EKS RBAC.

The Traditional-approach:

The Pulumi-approach:

Step 2: Create one EKS cluster. Validate cluster creation. Add the namespaces you need.

The Traditional-approach:

The Pulumi-approach:

Step 3: Understand Kubernetes RBAC. Declare the Kubernetes objects on the EKS cluster.

The Traditional-approach:

The Pulumi-approach:

Step 4: Update aws-iam-authenticator ConfigMap to map the IAM Roles to the Kubernetes usernames.

The Traditional-approach:

The Pulumi-approach:

Testing the Pulumi approach worked

Next Step

Using GitLab Pipelines with the six Pulumi stacks in `environment:dev` and `environment:prod`