Deploy the services-kafka helm chart on Google Kubernetes Engine (GKE)

Question

Pulumi · Accepted Answer

Deploying a Helm chart to a Google Kubernetes Engine (GKE) cluster involves several steps. You'll need to set up the GKE cluster, configure access to it, and then deploy the Helm chart to the cluster.

Below is a detailed explanation followed by a TypeScript program using Pulumi to accomplish your goal.

### Explanation

1. **Set up the GKE cluster**: You must have a Kubernetes cluster to deploy the Helm chart. We will use the `gcp.container.Cluster` resource to create a new GKE cluster.

2. **Configure k8s provider**: Once the cluster is provisioned, Pulumi needs to interact with it to deploy the Helm chart. We will configure the Kubernetes provider to point to the newly created cluster. This involves retrieving the cluster's `kubeconfig` which can be obtained from the GKE cluster's properties.

3. **Deploy the Helm chart**: You'll use the `helm.v3.Chart` resource from Pulumi's Kubernetes (k8s) package to deploy the "services-kafka" Helm chart onto the GKE cluster. The Helm resource takes in the name of the chart and other optional parameters to customize the deployment.

Now let's translate that into a Pulumi TypeScript program:

### Program

```typescript
import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";
import * as k8s from "@pulumi/kubernetes";

// Create a GKE cluster
const cluster = new gcp.container.Cluster("my-gke-cluster", {
    // Specify the required properties for your GKE cluster here
    initialNodeCount: 3,
    nodeVersion: "latest",
    minMasterVersion: "latest",
});

// Export the Cluster's name and Kubeconfig
export const kubeconfig = pulumi.
    all([cluster.name, cluster.endpoint, cluster.masterAuth]).
    apply(([name, endpoint, masterAuth]) => {
        const context = `${gcp.config.project}_${gcp.config.zone}_${name}`;
        return `apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: ${masterAuth.clusterCaCertificate}
    server: https://${endpoint}
  name: ${context}
contexts:
- context:
    cluster: ${context}
    user: ${context}
  name: ${context}
current-context: ${context}
kind: Config
preferences: {}
users:
- name: ${context}
  user:
    auth-provider:
      config:
        cmd-args: config config-helper --format=json
        cmd-path: gcloud
        expiry-key: '{.credential.token_expiry}'
        token-key: '{.credential.access_token}'
      name: gcp
`;
    });

// Create a Kubernetes provider instance that uses our cluster from above.
const k8sProvider = new k8s.Provider("my-k8s-provider", {
    kubeconfig: kubeconfig,
});

// Deploy the "services-kafka" Helm chart
const kafkaChart = new k8s.helm.v3.Chart("services-kafka", {
    chart: "kafka",
    version: "latest",
    // You might be using a different repository, make sure to include that in your `fetchOpts`
    fetchOpts: {repo: "https://charts.bitnami.com/bitnami"},
}, { provider: k8sProvider });

// Export the Helm Chart's status
export const kafkaStatus = kafkaChart.status;

```
This program starts by declaring the GKE cluster using `gcp.container.Cluster`. We provide a minimal configuration with the desired number of nodes and the Kubernetes version. It's important to adjust these properties based on your workload requirements.

The kubeconfig for accessing the GKE cluster is created by combining the cluster's name, endpoint, and authentication details. This kubeconfig is necessary for the Kubernetes provider and kubectl to communicate with the cluster.

The `k8s.Provider` resource uses the kubeconfig to establish a communication channel with the GKE cluster.

We then declare a Helm chart resource using `k8s.helm.v3.Chart`, which points to the "kafka" Helm chart in the Bitnami repository. We use the `provider` option to ensure Pulumi uses the correct Kubernetes provider instance that we've configured to communicate with the new GKE cluster.

Finally, we export the Helm chart's status to provide visibility into the deployment outcome.

Make sure to include all dependencies in your `package.json` file:

```json
{
  "dependencies": {
    "@pulumi/pulumi": "^3.0.0",
    "@pulumi/gcp": "^6.0.0",
    "@pulumi/kubernetes": "^3.0.0"
  }
}
```

After writing this code, you can deploy the resources by running `pulumi up`. The Pulumi CLI will interpret your TypeScript program and provision the resources in the correct order, handling all dependencies between them.

### Important Notes
- Ensure that you have the correct GCP credentials configured in Pulumi.
- The Bitnami Helm repository is used here as an example. If the Kafka chart is in a different repository, you will need to change the `fetchOpts` accordingly.
- You might need to provide additional configuration options for the Kafka Helm chart based on your specific requirements.
- The resources will incur costs in your Google Cloud account, so ensure to clean up resources using `pulumi destroy` if they are no longer needed.