Deploy the pinot helm chart on Azure Kubernetes Service (AKS)

Question

Pulumi · Accepted Answer

To deploy the Apache Pinot Helm chart on Azure Kubernetes Service (AKS), you need to perform the following high-level steps:

1. Set up an AKS cluster.
2. Install and configure Helm on your local machine or the environment where you're running Pulumi.
3. Use Pulumi's Kubernetes provider to deploy the Helm chart to your AKS cluster.

Below is a TypeScript program that uses Pulumi to create an AKS cluster and then deploy the Pinot Helm chart to that cluster. The program relies on the `azure-native` and `kubernetes` packages provided by Pulumi.

First, make sure you've [installed Pulumi](https://www.pulumi.com/docs/get-started/install/) and [configured your Azure credentials](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/).

Next, install the required Pulumi packages and the Kubernetes package using npm:

```bash
npm install @pulumi/pulumi @pulumi/azure-native @pulumi/kubernetes
```

Then, you can use the following TypeScript program to define your infrastructure and application deployment:

```typescript
import * as azure from "@pulumi/azure-native";
import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

// Create an Azure Resource Group
const resourceGroup = new azure.resources.ResourceGroup("myResourceGroup");

// Create an AKS cluster
const cluster = new azure.containerservice.ManagedCluster("myAKSCluster", {
    resourceGroupName: resourceGroup.name,
    agentPoolProfiles: [{
        count: 2,
        maxPods: 110,
        mode: "System",
        name: "agentpool",
        nodeLabels: {},
        osDiskSizeGB: 30,
        osType: "Linux",
        type: "VirtualMachineScaleSets",
        vmSize: "Standard_DS2_v2",
    }],
    dnsPrefix: "myakscluster",
    enableRBAC: true,
    kubernetesVersion: "1.18.14",
    linuxProfile: {
        adminUsername: "adminuser",
        ssh: {
            publicKeys: [{
                keyData: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC..."
            }],
        },
    },
    nodeResourceGroup: "myaksNodeResourceGroup",
});

// Export the cluster's kubeconfig
export const kubeconfig = pulumi.
    all([cluster.name, resourceGroup.name]).
    apply(([clusterName, rgName]) => {
        return azure.containerservice.listManagedClusterUserCredentials({
            resourceGroupName: rgName,
            resourceName: clusterName,
        });
    }).
    apply(creds => {
        const enc = new TextDecoder("utf-16");
        const kubeconfigs = creds.kubeconfigs!;
        if (kubeconfigs.length === 0) {
            return "";
        }
        return enc.decode(kubeconfigs[0].value);
    });

// Create a Kubernetes provider instance that uses our cluster from above.
const k8sProvider = new k8s.Provider("k8s", {
    kubeconfig: kubeconfig,
});

// Deploy the Pinot Helm chart using the Kubernetes provider
const pinotChart = new k8s.helm.v3.Chart("pinot", {
    chart: "pinot",
    version: "0.1.0", // specify the version of the chart
    fetchOpts: {
        repo: "https://raw.githubusercontent.com/apache/incubator-pinot/master/kubernetes/helm", // URL to the chart repository
    },
}, { provider: k8sProvider });

// Export the external IP of the Pinot service for easy access.
export const pinotExternalIp = pinotChart.getResourceProperty("v1/Service", "pinot-quickstart-broker", "status")
    .apply(status => status.loadBalancer.ingress[0].ip);
```

This program does the following:

- It creates a new Azure resource group to hold all resources.
- It defines an AKS cluster with a Linux node pool.
- It exports a kubeconfig that will be used to interact with the AKS cluster. You'll typically save this to a file and use it with kubectl or Helm.
- It establishes a Pulumi Kubernetes provider that points to the new AKS cluster using the obtained kubeconfig credentials.
- It deploys the Pinot Helm chart using the Kubernetes provider created earlier.
- It exports the external IP address of the Pinot service once deployed. You can use this IP to interact with your Pinot cluster.

To run this program, save it to a file (e.g., `index.ts`), and use the Pulumi CLI to create the necessary resources:

```bash
pulumi stack init dev
pulumi up
```
Remember to replace the placeholder SSH key in `keyData` with your actual SSH public key and set the correct Helm chart version and repository URL as needed.

This deployment may take several minutes to complete because it involves provisioning AKS cluster resources and deploying the Helm chart. Once complete, you will see stack outputs printed to the terminal, including the `kubeconfig` and `pinotExternalIp`, which you can use to interact with your AKS cluster and Apache Pinot.