Deploy the data-factory helm chart on Azure Kubernetes Service (AKS)

Question

Pulumi · Accepted Answer

To deploy a Helm chart for Azure Data Factory on an Azure Kubernetes Service (AKS) cluster, we will be using Pulumi's infrastructure as code approach. The process will involve creating an AKS cluster and then deploying the Data Factory chart on it using Pulumi's kubernetes package which provides a helm.sh/v3.Chart resource type for this purpose.

First, you'll need to have the Pulumi CLI installed and logged in. You'll also need to have an Azure account configured with the necessary permissions to create resources and Azure Kubernetes Service (AKS) enabled.

Here's a program in TypeScript that achieves your goal. It creates a new AKS cluster and deploys the Data Factory Helm chart on it. Below is a step-by-step guide, followed by the complete code:

Step-by-Step Guide:

Setup: We’ll use the azure-native package to create the AKS cluster. This is the Azure provider for Pulumi which allows us to manage Azure resources.
Cluster Creation: We create an AKS cluster by defining a ProvisionedCluster resource.
Data Factory Helm Chart Deployment: Once the cluster is provisioned, we use the Chart resource from the kubernetes/helm.sh/v3 package to deploy the Data Factory Helm chart.

Prerequisites:

Make sure you have the following prerequisites in place before running the program:

Install Node.js and npm.
Install Pulumi.
Configure Azure credentials with Pulumi using pulumi config set azure-native:clientId, pulumi config set azure-native:clientSecret, pulumi config set azure-native:tenantId, and pulumi config set azure-native:subscriptionId.

Detailed Pulumi Program:

import * as pulumi from "@pulumi/pulumi";
import * as azure from "@pulumi/azure";
import * as azuread from "@pulumi/azuread";
import * as kubernetes from "@pulumi/kubernetes";
import * as azureNative from "@pulumi/azure-native";

// Create an Azure Resource Group
const resourceGroup = new azureNative.resources.ResourceGroup("resourceGroup");

// Create an Azure AD application for the AKS cluster
const app = new azuread.Application("aks");

// Create a service principal for the application so AKS can act on behalf of the Azure account
const servicePrincipal = new azuread.ServicePrincipal("aksSp", {applicationId: app.applicationId});

// Generate a random password for the service principal
const servicePrincipalPassword = new azuread.ServicePrincipalPassword("aksSpPassword", {
    servicePrincipalId: servicePrincipal.id,
    value: "<password>", // Replace with a secure password.
    endDate: "2099-01-01T00:00:00Z",
});

// Now create the AKS cluster itself
const cluster = new azure.containerservice.KubernetesCluster("aksCluster", {
    resourceGroupName: resourceGroup.name,
    defaultNodePool: {
        name: "aksagentpool",
        nodeCount: 3,
        vmSize: azure.containerservice.KubernetesClusterDefaultNodePoolVmSize.StandardD2V2,
    },
    dnsPrefix: `${pulumi.getStack()}-kube`,
    linuxProfile: {
        adminUsername: "adminuser",
        sshKey: {
            keyData: "<ssh-rsa PUBLIC KEY>", // Replace with your SSH public key
        },
    },
    servicePrincipal: {
        clientId: app.applicationId,
        clientSecret: servicePrincipalPassword.value,
    },
    kubernetesVersion: "1.20.7",
});

// Export the kubeconfig for the cluster
export const kubeconfig = cluster.kubeConfigRaw;

// Create a provider for deploying resources to the AKS cluster
const k8sProvider = new kubernetes.Provider("k8sProvider", {
    kubeconfig: cluster.kubeConfigRaw,
});

// Deploy the Data Factory Helm chart using the AKS cluster's Kubernetes provider
const dataFactoryChart = new kubernetes.helm.v3.Chart("data-factory-chart", {
    chart: "data-factory",
    version: "1.0.0", // Replace with the desired chart version
    fetchOpts: {
        repo: "http://charts.example.com/", // Replace with the Data Factory chart's repository URL
    },
}, {provider: k8sProvider});

// Export the Data Factory chart's status
export const dataFactoryStatus = dataFactoryChart.status;

This will create both the AKS cluster and deploy the Data Factory Helm chart on it. To apply the code, save it to a file with a .ts extension, and run pulumi up.

Keep in mind that you need to replace placeholders like <password>, <ssh-rsa PUBLIC KEY>, and actual chart version and repository details for the Data Factory Helm chart in the code above with real values before running the program.