Deploy the kafka-connect-wrapper helm chart on Azure Managed Openshift Service

Question

Pulumi · Accepted Answer

To deploy the kafka-connect-wrapper Helm chart on Azure Managed OpenShift Service using Pulumi, you will need to follow these steps:

Set up an OpenShift Managed Cluster: You'll start by setting up an Azure Red Hat OpenShift (ARO) Managed Cluster using the azure-native.redhatopenshift.OpenShiftCluster resource.
Install and Use Helm Chart: After setting up the ARO cluster, you'll use the kubernetes.helm.v3.Chart resource to deploy the Helm chart for Kafka Connect.

Here's how you can accomplish this using Pulumi with TypeScript:

import * as pulumi from "@pulumi/pulumi";
import * as azure_native from "@pulumi/azure-native";
import * as kubernetes from "@pulumi/kubernetes";

// Step 1: Create an Azure Resource Group
const resourceGroup = new azure_native.resources.ResourceGroup("resourceGroup", {
    resourceGroupName: "openshiftResourceGroup",
    location: "East US", // You can choose the Azure region that is right for you
});

// Step 2: Create an Azure Red Hat OpenShift Cluster
const openshiftCluster = new azure_native.redhatopenshift.OpenShiftCluster("openshiftCluster", {
    resourceGroupName: resourceGroup.name,
    resourceName: "openshiftCluster", // Choose a name for your ARO cluster
    location: resourceGroup.location,
    clusterProfile: {
        domain: "example", // Replace with your own domain information
        resourceGroupId: resourceGroup.id,
        version: "4.6", // Specify the OpenShift version
        pullSecret: "<pullSecret>", // Provide the pull secret obtained from Red Hat OpenShift
    },
    masterProfile: {
        vmSize: "Standard_D8s_v3", // Specify the VM size for the master node
    },
    networkProfile: {
        podCidr: "10.42.0.0/16",
        serviceCidr: "10.41.0.0/16",
    },
    workerProfiles: [{
        name: "worker", // The name of the worker profile
        vmSize: "Standard_D4s_v3", // The VM size of the worker nodes
        diskSizeGB: 128, // Size of the disk attached to the workers
        count: 3, // The number of worker nodes
    }],
});

// Step 3: Deploy the kafka-connect-wrapper Helm chart
const provider = new kubernetes.Provider("k8sProvider", {
    kubeconfig: openshiftCluster.kubeadminConfig.raw, // Use the generated kubeconfig from the OpenShift cluster
});

const kafkaConnectChart = new kubernetes.helm.v3.Chart("kafka-connect-chart", {
    chart: "kafka-connect-wrapper",    // Name of the chart
    version: "1.0.0",                  // Version of the chart
    fetchOpts: {
        repo: "https://charts.bitnami.com/bitnami", // The Helm repository URL
    },
}, { provider: provider });

// Export the Kubernetes config and the service URL
export const kubeconfig = openshiftCluster.kubeadminConfig.raw;
export const serviceUrl = pulumi.interpolate`http://${kafkaConnectChart.getResourceProperty("v1/Service", "kafka-connect-chart-kafka-connect", "status").apply(status => status.loadBalancer.ingress[0].ip)}:80`;

This program does the following:

It creates a new Azure resource group to contain all related resources.
It then defines a new ARO cluster with the required profiles for master and worker nodes.
A Provider is created to allow Pulumi to interact with the OpenShift cluster using the obtained kubeconfig.
It deploys the kafka-connect-wrapper Helm chart on the ARO cluster using the Kubernetes provider.
Finally, the program exports the kubeconfig required to interact with the OpenShift cluster and the URL through which the Kafka Connect cluster can be accessed.

In this program, replace the placeholder values (for example <pullSecret>) with your actual values. The pull secret is used to authenticate with the Red Hat OpenShift registry and needs to be obtained from Red Hat.

Before running this Pulumi program, ensure you have the Pulumi CLI installed and logged in to both Azure and your Pulumi account. Then, you can execute the Pulumi program in your terminal:

Initialize a new Pulumi project.
Place the TypeScript code in your index.ts file.
Run pulumi up to create the resources.

Note that operations in Azure, such as creating an OpenShift cluster, can take some time. After the deployment, you will receive a URL for the Kafka Connect service which you can use to interact with your Kafka Connect instance.