Using kubernetes elbv2.k8s.aws with kibana.k8s.elastic.co

Question

Pulumi · Accepted Answer

If you want to set up a Kubernetes service on AWS and expose it using an Elastic Load Balancing (ELB) version 2, you have to create a Kubernetes cluster, deploy the specified services (like Kibana), and then create an ELB to manage the traffic. Pulumi offers various resources to handle such deployments; for Kubernetes cluster management, we can use the `eks.Cluster` component from the Pulumi EKS package, and for load balancing, we can use the `awsx.lb.ApplicationLoadBalancer` from the Pulumi AWSX package.

First, let's talk through the steps that we're going to take:

1. **Provision an EKS Cluster**: We'll start by creating a new cluster using the `eks.Cluster` component, which wraps the creation of an AWS EKS cluster with all of the necessary supporting resources such as the EC2 instances or managed node groups.
   
2. **Deploy Kibana**: With the cluster in place, we'll proceed to deploy Kibana into Kubernetes. Kibana will be configured as a Kubernetes Deployment, and we'll expose it using a Kubernetes Service.

3. **Create an Application Load Balancer**: Finally, we'll set up an ALB to balance traffic to Kibana. This involves creating a new `awsx.lb.ApplicationLoadBalancer` and configuring it to route traffic to our Kibana service.

Now, let's jump into the Pulumi TypeScript program which will execute the above plan:

```typescript
import * as eks from '@pulumi/eks';
import * as awsx from '@pulumi/awsx';
import * as k8s from '@pulumi/kubernetes';
import * as pulumi from '@pulumi/pulumi';

// Step 1: Provision an EKS Cluster
const cluster = new eks.Cluster('my-cluster', {
  desiredCapacity: 2,
  minSize: 1,
  maxSize: 2,
  storageClasses: "gp2", // Set the storage class to be used by the EKS cluster.
  version: "1.21", // Specify the version of Kubernetes.
});

// Step 2: Deploy Kibana
const kibanaAppLabels = { app: 'kibana' };
const kibanaDeployment = new k8s.apps.v1.Deployment('kibana-deployment', {
  metadata: {
    namespace: kubeSystemNamespaceName,
    labels: kibanaAppLabels,
  },
  spec: {
    replicas: 1,
    selector: { matchLabels: kibanaAppLabels },
    template: {
      metadata: {
        labels: kibanaAppLabels,
      },
      spec: {
        containers: [{
          name: 'kibana',
          image: 'docker.elastic.co/kibana/kibana:7.10.0', // Specify the Kibana docker image.
          ports: [{ containerPort: 5601 }], // Kibana runs on port 5601.
        }],
      },
    },
  },
}, { provider: cluster.provider });

const kibanaService = new k8s.core.v1.Service('kibana-service', {
  metadata: {
    labels: kibanaAppLabels,
  },
  spec: {
    type: 'LoadBalancer',
    selector: kibanaAppLabels,
    ports: [{ port: 80, targetPort: 5601 }], // Expose Kibana on port 80.
  },
}, { provider: cluster.provider });

// Step 3: Create an Application Load Balancer
const alb = new awsx.lb.ApplicationLoadBalancer('kibana-alb', {
  external: true, // We want the ALB to be internet-facing.
  securityGroups: cluster.clusterSecurityGroup.id, // Use the same security group as the EKS cluster.
  listeners: [{
    port: 80, // Listen to HTTP traffic.
    defaultActions: [{
      type: 'forward',
      targetGroup: alb.defaultTargetGroup,
    }],
  }],
});

// Export the URL of the ALB so we can access Kibana from the browser.
export const kibanaUrl = alb.loadBalancer.dnsName;

// Now the Kibana instance is up and reachable through ALB at the exported URL.
```

Let's disseminate the above Pulumi code.

- We create an EKS cluster with the necessary configurations, including the version of Kubernetes to use and the desired number of nodes.
  
- We set up the Kibana service. It's wrapped in a deployment object, which ensures that a specified number of replicas of Kibana pods are always running. We also create a service object to expose Kibana and its port.

- We provision an AWS Application Load Balancer (ALB) that listens on port 80 and forwards traffic to the Kibana service within the Kubernetes cluster.

- Finally, we export the DNS name of the ALB as `kibanaUrl`. Accessing Kibana would then be as simple as navigating to that URL in a web browser.

The above script expects you to have AWS and Pulumi access configured on the machine where it's being run. It also uses the default VPC and subnets of your AWS account to place all the resources unless otherwise specified.

You would run this program using the Pulumi CLI as follows:

```bash
pulumi up
```

This command provisions the resources defined in your program, providing a preview before making any changes and waiting for your confirmation before proceeding.

Upon successful deployment, Pulumi will output the `kibanaUrl` which you can use to access the Kibana dashboard on your web browser.