Deploy the airbyte-api-server helm chart on Azure Kubernetes Service (AKS)

Question

Pulumi · Accepted Answer

To deploy the `airbyte-api-server` Helm chart on Azure Kubernetes Service (AKS), you will need to perform several steps:

1. Set up an AKS cluster. Azure Kubernetes Service (AKS) manages your hosted Kubernetes environment, making it easier to deploy, manage, and scale containerized applications using Kubernetes without in-depth knowledge of Kubernetes.

2. Install and configure Helm on your machine. Helm is a package manager for Kubernetes that allows you to define, install, and upgrade even the most complex Kubernetes applications.

3. Add the Helm repository that contains the `airbyte-api-server` chart.

4. Use Helm to deploy the chart to your AKS cluster.

Here is a detailed Pulumi TypeScript program that sets up an AKS cluster and deploys the `airbyte-api-server` Helm chart. The Pulumi Kubernetes provider package includes a `Chart` resource that represents a Helm chart for deployment.

To run this code, ensure you have logged in to Azure using the Azure CLI (`az login`) and have Pulumi correctly [installed and set up](https://www.pulumi.com/docs/get-started/azure/).

Now follow the TypeScript program below.

```typescript
import * as azure from "@pulumi/azure-native";
import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";
import * as random from "@pulumi/random";

// Create a resource group for AKS
const resourceGroup = new azure.resources.ResourceGroup("myResourceGroup");

// Create an AKS cluster
const aksCluster = new azure.containerservice.KubernetesCluster("myAKSCluster", {
    resourceGroupName: resourceGroup.name,
    defaultNodePool: {
        name: "aksagentpool",
        nodeCount: 2,
        vmSize: "Standard_DS2_v2",
    },
    dnsPrefix: `${pulumi.getStack()}-kube`,
    linuxProfile: {
        adminUsername: "aksuser",
        sshKey: {
            keyData: new random.RandomString("ssh-key", {
                length: 2048,
                special: false,
            }).result,
        },
    },
    servicePrincipal: {
        clientId: "YOUR_AZURE_CLIENT_ID",
        clientSecret: "YOUR_AZURE_CLIENT_SECRET",
    },
});

// Export the Kubeconfig
export const kubeconfig = pulumi.
    all([aksCluster.name, resourceGroup.name]).
    apply(([clusterName, rgName]) => {
        return azure.containerservice.listManagedClusterUserCredentials({
            resourceGroupName: rgName,
            resourceName: clusterName,
        });
    }).
    apply(creds => {
        const encoded = creds.kubeconfigs[0].value;
        return Buffer.from(encoded, 'base64').toString();
    });

// Create a Kubernetes provider instance that uses our cluster from above.
const k8sProvider = new k8s.Provider("k8s-provider", {
    kubeconfig: kubeconfig,
});

// Add the Helm repository
const airbyteRepo = "https://charts.airbyte.io";

// Deploy the airbyte-api-server Helm chart
const airbyteChart = new k8s.helm.v3.Chart("airbyte-api-server", {
    chart: "airbyte",
    version: "0.1.0", // Replace with the version you want to deploy
    fetchOpts: {
        repo: airbyteRepo,
    },
}, { provider: k8sProvider });

// The program **requires** you to have `airbyte-api-server` Helm chart available in the
// `https://charts.airbyte.io` repository, and you should replace `version` with the actual
// chart version you want to deploy.

// Export the URL to the airbyte application
// Here, you would need to customize the export line depending on how the `airbyte-api-server` Helm chart
// exposes the service (e.g., as a LoadBalancer, NodePort, etc.). For example, if it's available on a
// LoadBalancer, get the IP like so:
export const airbyteApiServerUrl = airbyteChart.getResourceProperty("v1/Service", "airbyte-api-server", "status")
    .apply(status => `http://${status.loadBalancer.ingress[0].ip}`);
```

This code performs the following actions:

1. **Resource Group Creation**: Creates a new Azure resource group which will contain all our resources.

2. **AKS Cluster Creation**: Provision a new AKS cluster in the resource group with two nodes using the `Standard_DS2_v2` virtual machine size.

3. **SSH Key Generation**: For the `linuxProfile` we create an SSH key that will be used to access the cluster nodes if needed.

4. **Service Principal**: This needs to be updated with your Azure Client ID and Client Secret to create the cluster with the correct permissions.

5. **Kubeconfig Export**: Retrieves the kubeconfig from the created AKS cluster, which will allow you to interact with your cluster through `kubectl`.

6. **Kubernetes Provider**: Configures the Pulumi Kubernetes provider to use the kubeconfig from our AKS cluster.

7. **Helm Chart Repository**: Points to the Helm repository where the `airbyte-api-server` chart is located.

8. **Deploy Helm Chart**: Deploys the `airbyte-api-server` Helm chart to the AKS cluster with the Kubernetes provider.

9. **Service URL Export**: Exports the URL where the `airbyte-api-server` can be accessed after it's deployed. This assumes that your service is of type `LoadBalancer`. If your service is of a different type, such as `NodePort`, you will need to handle it accordingly.

Replace `YOUR_AZURE_CLIENT_ID` and `YOUR_AZURE_CLIENT_SECRET` with your actual Azure credentials. These are required for Pulumi to authenticate with Azure and create resources on your behalf.

After this Pulumi program is executed, you will have a running AKS cluster with the `airbyte-api-server` deployed. You can then use the exported URL to access the Airbyte API server.