Using kubernetes awsloadbalancercontroller with k8s.cni.cncf.io

Question

Pulumi · Accepted Answer

When you want to use AWS Load Balancer Controller (formerly known as AWS ALB Ingress Controller) with Kubernetes, you typically do so in an Amazon EKS (Elastic Kubernetes Service) cluster that has the AWS CNI (Amazon VPC CNI plugin for Kubernetes) already set up. The AWS Load Balancer Controller enables you to manage AWS Elastic Load Balancers (like the Application Load Balancer) for a Kubernetes cluster.

The AWS Load Balancer Controller handles the creation, updating, and deletion of AWS Elastic Load Balancers as you create, update, or delete Ingress objects or Service objects of type LoadBalancer in your Kubernetes cluster. It relies on the k8s.cni.cncf.io networking annotations to understand the networking setup of the cluster managed by AWS CNI.

Below is a simple Pulumi program written in TypeScript that demonstrates how to set up an AWS Load Balancer Controller on an EKS cluster. We'll use the @pulumi/aws and @pulumi/eks packages for this purpose.

First, ensure you have Pulumi installed and your AWS account set up with the necessary credentials. Then, you can create a new Pulumi project:

pulumi new aws-typescript

Now, you can use the following program, which performs these tasks:

Creates an EKS cluster.
Sets up the AWS Load Balancer Controller by installing necessary RBAC (Role-Based Access Control) and the Helm chart.

import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Step 1: Create an EKS cluster.
const cluster = new eks.Cluster("my-cluster", {
    instanceType: "t2.medium",
    desiredCapacity: 2,
    minSize: 1,
    maxSize: 3,
    providerCreds: new pulumi.FileAsset("aws-creds.json"), // replace with your AWS credentials file
});

// Export the cluster's kubeconfig and name
export const kubeconfig = cluster.kubeconfig;
export const clusterName = cluster.eksCluster.name;

// Step 2: Install the AWS Load Balancer Controller.
// We will need to install a specific version compatible with our EKS cluster version.
const awsLoadBalancerControllerVersion = "v2.2.0"; // check the AWS documentation for the version that suits your EKS cluster

// This is the namespace where the AWS Load Balancer Controller will be installed.
const namespace = new k8s.core.v1.Namespace("aws-loadbalancer", {
    metadata: {
        name: "kube-system", // typically installed in the kube-system namespace
    },
}, { provider: cluster.provider });

// RBAC setup for the AWS Load Balancer Controller.
const serviceAccount = new k8s.core.v1.ServiceAccount("aws-loadbalancer-controller-sa", {
    metadata: {
        namespace: namespace.metadata.name,
        name: "aws-loadbalancer-controller",
        annotations: {
            "eks.amazonaws.com/role-arn": "arn:aws:iam::<ACCOUNT_ID>:role/AWSLoadBalancerControllerIAMRole", // replace <ACCOUNT_ID> with your AWS account ID
        },
    },
}, { provider: cluster.provider });

// Install the Helm chart for the AWS Load Balancer Controller.
const chart = new k8s.helm.v3.Chart("aws-loadbalancer-controller", {
    namespace: namespace.metadata.name,
    chart: "aws-load-balancer-controller",
    version: "1.2.3", // make sure to use the version that matches the 'awsLoadBalancerControllerVersion'
    fetchOpts: {
        repo: "https://aws.github.io/eks-charts",
    },
    values: {
        clusterName: clusterName, // passes the cluster name to the controller configuration
        serviceAccount: {
            // we define the service account details that our controller will use
            create: false,
            name: "aws-loadbalancer-controller",
        },
    }
}, { provider: cluster.provider });

// Exporting some details about the installed AWS Load Balancer Controller.
export const awsLoadBalancerControllerName = chart.resources.apply(r => r["kubernetes:apps/v1:Deployment::aws-loadbalancer-controller"].metadata.name);
export const awsLoadBalancerControllerNamespace = namespace.metadata.name;

Explanation:

We create an Amazon EKS cluster named "my-cluster" using the eks.Cluster class. We have set a specific instance type, desired capacity, and other scaling parameters.
We create a Kubernetes namespace in which the AWS Load Balancer Controller will reside (usually kube-system) using k8s.core.v1.Namespace.
We create a Kubernetes service account with the name "aws-loadbalancer-controller" using k8s.core.v1.ServiceAccount, which the AWS Load Balancer Controller will use. It contains an annotation specifying the ARN of the IAM role that grants it necessary permissions. You need to replace <ACCOUNT_ID> with your actual AWS account ID, and the role should be prepared as per AWS's documentation.
We install the AWS Load Balancer Controller using Helm and the k8s.helm.v3.Chart class, pointing to the EKS charts repository, with the specific version of the controller that works with our EKS cluster.
We export the name of the AWS Load Balancer Controller deployment and its namespace for quick reference.

Make sure to install @pulumi/eks and @pulumi/kubernetes via npm if you haven't already:

npm install @pulumi/eks @pulumi/kubernetes

Make sure also to replace "aws-creds.json" with the path to your AWS credentials file, <ACCOUNT_ID> with your AWS account ID, and "AWSLoadBalancerControllerIAMRole" with the name of the IAM role you have set up for the AWS Load Balancer Controller.

Before deploying this, go through the AWS Load Balancer Controller documentation to make sure you have all the prerequisites ready. This includes setting up the right IAM roles and policies.

Once you're all set, you can run the following command to deploy the application:

pulumi up

This will provision the necessary resources on AWS, and deploy the AWS Load Balancer Controller on your EKS cluster. You can then create Ingress objects or Services of type LoadBalancer and the AWS Load Balancer Controller will manage the corresponding AWS Elastic Load Balancers for you.