Deploy the k8s-spot-rescheduler helm chart on Google Kubernetes Engine (GKE)

Question

Pulumi · Accepted Answer

To deploy the `k8s-spot-rescheduler` Helm chart on Google Kubernetes Engine (GKE), you'll need to perform the following steps:

1. **Create a GKE Cluster:** Before you can deploy the Helm chart, you need a Kubernetes cluster. You will use Pulumi to provision a new GKE cluster.

2. **Configure kubectl:** Once the cluster is created, you need to configure `kubectl` to interact with the new cluster.

3. **Install Helm:** Helm is a package manager for Kubernetes, which you will use to install the `k8s-spot-rescheduler` chart.

4. **Deploy the Helm Chart:** With Helm installed and configured, you deploy the chart to your GKE cluster.

Below is a Pulumi program written in TypeScript that performs these steps. The program uses the `gcp` package to create GKE cluster resources and the `kubernetes` package for Helm chart deployment.

```typescript
import * as gcp from "@pulumi/gcp";
import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

// Step 1: Create a GKE cluster
const cluster = new gcp.container.Cluster("gke-cluster", {
    initialNodeCount: 2,
    minMasterVersion: "latest",
    nodeVersion: "latest",
    nodeConfig: {
        preemptible: true,
        machineType: "n1-standard-1",
        oauthScopes: [
            "https://www.googleapis.com/auth/compute",
            "https://www.googleapis.com/auth/devstorage.read_only",
            "https://www.googleapis.com/auth/logging.write",
            "https://www.googleapis.com/auth/monitoring"
        ],
    },
});

// Export the Cluster name
export const clusterName = cluster.name;

// Obtain the Kubeconfig after cluster creation to interact with the cluster
const kubeconfig = pulumi.
    all([cluster.name, cluster.endpoint, cluster.masterAuth]).
    apply(([name, endpoint, masterAuth]) => {
        const context = `${gcp.config.project}_${gcp.config.zone}_${name}`;
        return `apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: ${masterAuth.clusterCaCertificate}
    server: https://${endpoint}
  name: ${context}
contexts:
- context:
    cluster: ${context}
    user: ${context}
  name: ${context}
current-context: ${context}
kind: Config
preferences: {}
users:
- name: ${context}
  user:
    auth-provider:
      config:
        cmd-args: config config-helper --format=json
        cmd-path: gcloud
        expiry-key: '{.credential.token_expiry}'
        token-key: '{.credential.access_token}'
      name: gcp
`;
    });

// Step 2: Configure the Kubernetes provider to use the generated kubeconfig
const k8sProvider = new k8s.Provider("gke-k8s", {
    kubeconfig: kubeconfig,
});

// Step 3: Install the Helm chart for the k8s-spot-rescheduler
const spotReschedulerChart = new k8s.helm.v3.Chart("k8s-spot-rescheduler", {
    chart: "k8s-spot-rescheduler",
    version: "1.0.0", // replace with actual chart version
    fetchOpts: {
        repo: "http://charts.example.com/", // replace with the actual Helm repo URL
    }
}, { provider: k8sProvider });

// Export the Helm chart name
export const spotReschedulerChartName = spotReschedulerChart.releaseName;

```

Here's what each part of the program does:

- **GKE Cluster Creation:** We create a GKE cluster with a predefined number of nodes. Preemptible nodes are chosen here because the `k8s-spot-rescheduler` is often used in combination with preemptible nodes to help reschedule pods when preemptible instances are reclaimed.

- **Kubeconfig Generation:** After the GKE cluster is provisioned, we generate a kubeconfig file that `kubectl` and Pulumi can use to interact with the cluster.

- **Kubernetes Provider Configuration:** This tells Pulumi how to communicate with the new GKE cluster. It uses the kubeconfig from the previous step.

- **Helm Chart Deployment:** We define a Helm chart resource to deploy. You should replace the `chart`, `version`, and `repo` with the actual details of the `k8s-spot-rescheduler` Helm chart.

To run this program, you'll need to have Pulumi installed and set up with the appropriate GCP credentials. Then, you can run `pulumi up` to perform the deployment. The program will provision the resources in the correct order and output the names of the created GKE cluster and Helm deployment.