Deploy the elastic-filebeat helm chart on AWS EKS

Question

Pulumi · Accepted Answer

To deploy the elastic-filebeat Helm chart on an AWS EKS cluster using Pulumi, you'll need to follow these steps:

1. Create an EKS cluster: Provision an Amazon EKS cluster where your Elastic Filebeat will run.
2. Deploy the Helm chart: Once you have the cluster, deploy the Elastic Filebeat Helm chart to the EKS cluster using Pulumi's Helm chart resource.

Here's a detailed Pulumi program written in TypeScript that demonstrates these steps:

### Step-by-Step Program Explanation and Definition

#### 1. Provision an EKS Cluster
We'll use the `eks.Cluster` resource from the Pulumi EKS package. This high-level component creates and configures an AWS EKS Kubernetes cluster for us with sensible defaults.

- `vpcId` and `subnetIds` are needed to specify the VPC configuration for the EKS cluster.
- `instanceType` specifies the EC2 instance type for the EKS worker nodes.
- `desiredCapacity`, `minSize`, and `maxSize` control the scaling properties of your worker nodes.

#### 2. Deploy the Elastic Filebeat Helm chart
For the Elastic Filebeat Helm chart deployment, we use the `Chart` resource from Pulumi's Kubernetes provider. It will install the Helm chart into the EKS cluster we just provisioned.

- Specify the chart name, `elastic/filebeat`.
- The `values` property can override the default chart values, including the Elastic search host, protocol, and other Filebeat configurations as needed.

Here's the complete program:

```typescript
import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";

// Create an EKS cluster.
const cluster = new eks.Cluster("eks-cluster", {
    instanceType: "t2.medium",
    desiredCapacity: 2,
    minSize: 1,
    maxSize: 3,
});

// Create a Kubernetes provider instance using the kubeconfig from the EKS cluster.
const provider = new k8s.Provider("eks-k8s", {
    kubeconfig: cluster.kubeconfig.apply(JSON.stringify),
});

// Deploy the Elastic Filebeat Helm chart to the EKS cluster.
const elasticFilebeat = new k8s.helm.v3.Chart("elastic-filebeat", {
    chart: "filebeat",
    version: "7.10.0", // specify the version of Filebeat you want to deploy
    fetchOpts: {
        repo: "https://helm.elastic.co",
    },
    values: {
        elasticsearch: {
            host: "elasticsearch-host.yourdomain.com", // replace with your ElasticSearch endpoint
            protocol: "https",
        },
    },
}, { provider });

// Export the kubeconfig to access the EKS cluster.
export const kubeConfig = cluster.kubeconfig;

// Export the Filebeat service URL, in case it’s exposed via a LoadBalancer or NodePort.
export const filebeatServiceUrl = elasticFilebeat.getResourceProperty("v1/Service", "elastic-filebeat-filebeat", "status").apply(status => status.loadBalancer?.ingress[0]?.ip || status.loadBalancer?.ingress[0]?.hostname);
```

### Explanation of Pulumi Program

This Pulumi program first creates an EKS cluster and then deploys the Elastic Filebeat Helm chart onto it. The EKS cluster is provisioned with default settings suitable for a small to medium-scale application. After the cluster creation, Pulumi programmatically obtains the kubeconfig, which will be needed by the Kubernetes provider to interact with the EKS cluster.

The program then proceeds to define the Elastic Filebeat Helm chart deployment using the corresponding chart from the Elastic Helm repository. The `values` section within the Helm chart definition is used to customize the deployment, such as specifying the ElasticSearch host and protocol details.

Lastly, the code exports the kubeconfig and the service URL for Filebeat, which can be used to interact with the Kubernetes cluster and the Filebeat service respectively.

Please ensure that you have Pulumi installed and configured with AWS credentials before running this program. To deploy this stack, navigate to the directory containing this Pulumi program, run `pulumi up`, and confirm the deployment when prompted.