Deploy the kafka-helm-chart helm chart on Digital Ocean Kubernetes Service

Question

Pulumi · Accepted Answer

To deploy the kafka-helm-chart on a Digital Ocean Kubernetes Service using Pulumi, we will go through a few steps:

1. We'll create a new Kubernetes cluster on Digital Ocean using the `digitalocean.KubernetesCluster` resource.
2. We'll set up the Helm chart for Kafka using the `kubernetes.helm.v3.Chart` resource from the Kubernetes provider.
3. We'll provide the necessary configuration for the Kafka Helm chart.

Let's start by creating a new Digital Ocean Kubernetes cluster:

```typescript
import * as digitalocean from "@pulumi/digitalocean";
import * as kubernetes from "@pulumi/kubernetes";

// Create a Digital Ocean Kubernetes cluster
const cluster = new digitalocean.KubernetesCluster("do-cluster", {
    region: "nyc1",
    version: "latest", // Specify the version for the cluster, you can also pin to a specific version
    nodePool: {
        name: "default",
        size: "s-2vcpu-2gb", // Choose the type of Droplet that fits your needs
        nodeCount: 3, // Number of nodes in the node pool
    },
});

// Export the cluster kubeconfig
export const kubeconfig = cluster.kubeConfigs[0].rawConfig;
```

In the above code, we have created a Kubernetes cluster in the `nyc1` region, with a single node pool that has three nodes of size `s-2vcpu-2gb`. We are telling Pulumi to use the `latest` Kubernetes version that Digital Ocean provides. Adjust the `region`, `version`, `size`, and `nodeCount` as necessary for your setup.

Once the cluster is provisioned, we'll use the `kubeconfig` to configure the Kubernetes provider and deploy Kafka:

```typescript
// Configure the Kubernetes provider with the kubeconfig from the DO cluster
const k8sProvider = new kubernetes.Provider("do-k8s", {
    kubeconfig: cluster.kubeConfigs[0].rawConfig,
});

// Deploy kafka-helm-chart using Helm
const kafkaChart = new kubernetes.helm.v3.Chart("kafka", {
    chart: "kafka",
    version: "8.19.0", // Replace with desired chart version
    fetchOpts: {
        repo: "https://charts.bitnami.com/bitnami", // Bitnami Helm repository
    },
    namespace: "default",
}, { provider: k8sProvider });

// Export the Helm chart deployment name
export const kafkaChartName = kafkaChart.releaseName;
```

This block of code sets up the Pulumi Kubernetes provider to work with our newly created cluster. All operations by this provider will happen against our Digital Ocean cluster. We then create a new Helm chart resource representing the Kafka chart. Make sure to use the correct `version` for the Helm chart you want to deploy, and validate that the repository URL (`repo`) points to the official Kafka Helm chart repository, if necessary.

Please make sure to replace `8.19.0` with the desired chart version that you want to install. By using `export`, we are exposing imperative details of the deployment, such as the cluster `kubeconfig` and the Kafka Helm chart release name, which can be used to interact with the cluster after the chart is deployed.

Remember to install the Pulumi CLI, configure your Pulumi credentials, and set up Digital Ocean access for Pulumi on your machine. If you haven't done this yet, refer to the [Pulumi documentation](https://www.pulumi.com/docs/get-started/digitalocean/) to get started.

After configuring your Pulumi environment, you can run the following commands to deploy your resources:

- Run `pulumi up` to preview and deploy the changes.
- Check the deployed resources in the Digital Ocean dashboard or using `kubectl` with the exported `kubeconfig`.
- To delete the resources, you can run `pulumi destroy`.

That's it—you now have a Kafka deployment running on a Digital Ocean Kubernetes Service managed by Pulumi!