Deploy the Dolphinscheduler Helm Chart on TypeScriptogle Kubernetes Engine (GKE)

Introduction

In this guide, we will deploy the DolphinScheduler Helm chart on a Google Kubernetes Engine (GKE) cluster using Pulumi with TypeScript. DolphinScheduler is a distributed and extensible workflow scheduler platform. We will leverage Pulumi to manage our infrastructure as code, ensuring a repeatable and consistent deployment process.

Step-by-Step Explanation

Step 1: Set Up Pulumi and GKE

1. Install Pulumi CLI: If you haven’t already, install the Pulumi CLI from here.
2. Authenticate with Google Cloud: Ensure you have authenticated with Google Cloud SDK by running gcloud auth login.
3. Create a New Pulumi Project: Initialize a new Pulumi project using pulumi new typescript.

Step 2: Configure GKE Cluster

1. Create a GKE Cluster: Define the GKE cluster configuration in your Pulumi program.
2. Obtain Kubernetes Credentials: Ensure your Pulumi program can obtain the necessary Kubernetes credentials to interact with the GKE cluster.

Step 3: Deploy DolphinScheduler Helm Chart

1. Add Helm Chart Repository: Add the Helm chart repository for DolphinScheduler.
2. Deploy the Helm Chart: Use Pulumi’s Helm provider to deploy the DolphinScheduler chart to your GKE cluster.

Summary

By following this guide, you have successfully deployed the DolphinScheduler Helm chart on a GKE cluster using Pulumi with TypeScript. This approach leverages Pulumi’s infrastructure as code capabilities to ensure a consistent and repeatable deployment process.

Full Code Example

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";
import * as k8s from "@pulumi/kubernetes";

// Create a GKE cluster
const cluster = new gcp.container.Cluster("dolphin-cluster", {
    initialNodeCount: 3,
    minMasterVersion: "1.21",
    nodeVersion: "1.21",
    nodeConfig: {
        machineType: "e2-medium",
        oauthScopes: [
            "https://www.googleapis.com/auth/cloud-platform",
        ],
    },
});

// Export the Kubeconfig
export const kubeconfig = pulumi.all([cluster.name, cluster.endpoint, cluster.masterAuth]).apply(
    ([name, endpoint, auth]) => {
        const context = \`\${gcp.config.project}_\${gcp.config.zone}_\${name}\`;
        return \`apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: \${auth.clusterCaCertificate}
    server: https://\${endpoint}
  name: \${context}
contexts:
- context:
    cluster: \${context}
    user: \${context}
  name: \${context}
current-context: \${context}
kind: Config
preferences: {}
users:
- name: \${context}
  user:
    auth-provider:
      config:
        cmd-args: config config-helper --format=json
        cmd-path: gcloud
        token-key: '{.credential.access_token}'
      name: gcp
\`;
    });

// Create a Kubernetes provider instance using the Kubeconfig
const k8sProvider = new k8s.Provider("k8s-provider", {
    kubeconfig: kubeconfig,
});

// Deploy the DolphinScheduler Helm chart
const dolphinSchedulerRelease = new k8s.helm.v3.Chart("dolphin-scheduler", {
    chart: "dolphinscheduler",
    version: "1.3.6",
    fetchOpts: {
        repo: "https://apache.github.io/dolphinscheduler-helm-chart",
    },
}, { provider: k8sProvider });

export const dolphinSchedulerReleaseName = dolphinSchedulerRelease.getResource("v1/Service", "dolphin-scheduler").metadata.name;